{"id":1229,"date":"2021-05-16T10:33:55","date_gmt":"2021-05-16T08:33:55","guid":{"rendered":"https:\/\/www.netexpertise.eu\/en\/?p=1229"},"modified":"2021-05-16T15:48:24","modified_gmt":"2021-05-16T13:48:24","slug":"mysql-postgresql-on-iscsi-fail-to-start-at-boot","status":"publish","type":"post","link":"http:\/\/www.netexpertise.eu\/en\/systems\/linux\/mysql-postgresql-on-iscsi-fail-to-start-at-boot.html","title":{"rendered":"MySQL \/ PostgreSQL on iSCSI Fail to Start at Boot"},"content":{"rendered":"\n<p>You are hosting Mysql or PostgreSQL data directory on <a href=\"\/en\/systems\/linux\/multipathed-iscsi-san-disks-on-linux.html\">iSCSI disks<\/a> but the service fails to start at server&#8217;s boot. The service does not find the directory. However, you can start the service manually if you log on the server once SSH is available.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" src=\"\/images\/iscsi_db_start.png\" alt=\"Mysql \/ PostgreSQL on iSCSI do not start\"\/><\/figure>\n\n\n\n<p>These are logs for Mariadb but they would be similar for Mysql:<br><br><\/p>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"vim\" class=\"language-vim\">mariadbd[795]: 0 [Note] \/usr\/sbin\/mysqld (mysqld 10.5.9-MariaDB-1:10.5.9+maria~buster-log) starting as process 795 ...\nmariadbd[795]: 0 [Warning] Can't create test file \/opt\/db\/data\/database_server.lower-test\nmariadbd[795]: #007\/usr\/sbin\/mysqld: Cannot change dir to '\/var\/lib\/mysql\/data\/' (Errcode: 2 \"No such file or directory\")\nmariadbd[795]: 0 [ERROR] Aborting\nsystemd[1]: mariadb.service: Main process exited, code=exited, status=1\/FAILURE\nsystemd[1]: mariadb.service: Failed with result 'exit-code'.\nsystemd[1]: Failed to start MariaDB 10.5.9 database server.<\/code><\/pre>\n\n\n\n<p><br>You may also get logs similar to these for PostgreSQL. I stored PostgreSQL on a XFS partition, hosted on <a href=\"https:\/\/opensource.com\/business\/16\/9\/linux-users-guide-lvm\" target=\"_blank\" rel=\"noreferrer noopener\">LVM<\/a> for a more flexible disk space management, such as adding space while the filesystem is mounted. This is probably the main reason why iSCSI disks are popular.<br><br><\/p>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"vim\" class=\"language-vim\">systemd: mounting \/var\/lib\/pgsql\nstarting PostgreSQL database server\nsd 2:0:0:0: [sdb] attached SCSI disk\nxfs (dm-4): Mounting V4 Filesystem\npostgresql-check-db-dir: \"\/var\/lib\/pgsql\/data\" is missing or empty\npostgresql.service: control process exited, code=exited status=1\nFailed to start PostgreSQL database server.<\/code><\/pre>\n\n\n\n<p><br>The database starting manually after the boot indicates there&#8217;s most likely a problem with the services boot order. Databases should start <span style=\"text-decoration: underline;\">after<\/span> iscsi disks are made available. You can solve this issue adding &#8220;<em>After=remote-fs.target<\/em>&#8221; in the service systemd file such as \/usr\/lib\/systemd\/system\/postgresql-9.5.service for PosgreSQL for instance. This is a way to manage service precedence and dependencies.<br><br>Note you may lose these changes next time the package is upgraded. Systemd lets you create an extra file where you can define your own settings. It will never modify it since this is your own file.<br>Just create \/etc\/systemd\/system\/mariadb.service.d\/override.conf as follow for Mariadb, it is the same process for Mysql or PostgreSQL:<br><br><\/p>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"vim\" class=\"language-vim\">[Service]\nEnvironment=\"UMASK_DIR=0750\"\n\n[Unit]\nAfter=remote-fs.target<\/code><\/pre>\n\n\n\n<p><br>In this file, I also changed the default data directory permissions so users in Mysql group can walk into it.<br>Run <em>systemctl daemon-reload<\/em> so it takes the new settings into account and reboot the server. The database service should now start.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>You are hosting Mysql or PostgreSQL data directory on iSCSI disks but the service fails to start at server&#8217;s boot. The service does not find the directory. However, you can start the service manually if you log on the server once SSH is available. These are logs for Mariadb but they would be similar for [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0},"categories":[11,4,25],"tags":[268,386,441,385,15],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v21.8.1 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Netexpertise - MySQL \/ PostgreSQL on iSCSI Fail to Start at Boot<\/title>\n<meta name=\"description\" content=\"Mysql and PostgreSQL on iSCSI disks fail to start at boot. Have you checked dependencies and the order in which systemd services start?\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.netexpertise.eu\/en\/systems\/linux\/mysql-postgresql-on-iscsi-fail-to-start-at-boot.html\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Netexpertise - MySQL \/ PostgreSQL on iSCSI Fail to Start at Boot\" \/>\n<meta property=\"og:description\" content=\"Mysql and PostgreSQL on iSCSI disks fail to start at boot. Have you checked dependencies and the order in which systemd services start?\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.netexpertise.eu\/en\/systems\/linux\/mysql-postgresql-on-iscsi-fail-to-start-at-boot.html\" \/>\n<meta property=\"og:site_name\" content=\"Netexpertise\" \/>\n<meta property=\"article:published_time\" content=\"2021-05-16T08:33:55+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2021-05-16T13:48:24+00:00\" \/>\n<meta name=\"author\" content=\"dave\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@netexpertise\" \/>\n<meta name=\"twitter:site\" content=\"@netexpertise\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.netexpertise.eu\/en\/systems\/linux\/mysql-postgresql-on-iscsi-fail-to-start-at-boot.html\",\"url\":\"https:\/\/www.netexpertise.eu\/en\/systems\/linux\/mysql-postgresql-on-iscsi-fail-to-start-at-boot.html\",\"name\":\"Netexpertise - MySQL \/ PostgreSQL on iSCSI Fail to Start at Boot\",\"isPartOf\":{\"@id\":\"https:\/\/www.netexpertise.eu\/en\/#website\"},\"datePublished\":\"2021-05-16T08:33:55+00:00\",\"dateModified\":\"2021-05-16T13:48:24+00:00\",\"author\":{\"@id\":\"https:\/\/www.netexpertise.eu\/en\/#\/schema\/person\/cb4cd666549d22e9070ec1cfc1a496fa\"},\"description\":\"Mysql and PostgreSQL on iSCSI disks fail to start at boot. Have you checked dependencies and the order in which systemd services start?\",\"breadcrumb\":{\"@id\":\"https:\/\/www.netexpertise.eu\/en\/systems\/linux\/mysql-postgresql-on-iscsi-fail-to-start-at-boot.html#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.netexpertise.eu\/en\/systems\/linux\/mysql-postgresql-on-iscsi-fail-to-start-at-boot.html\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.netexpertise.eu\/en\/systems\/linux\/mysql-postgresql-on-iscsi-fail-to-start-at-boot.html#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.netexpertise.eu\/en\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"MySQL \/ PostgreSQL on iSCSI Fail to Start at Boot\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.netexpertise.eu\/en\/#website\",\"url\":\"https:\/\/www.netexpertise.eu\/en\/\",\"name\":\"Netexpertise\",\"description\":\"Systems \/ Networks \/ DevOps\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.netexpertise.eu\/en\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.netexpertise.eu\/en\/#\/schema\/person\/cb4cd666549d22e9070ec1cfc1a496fa\",\"name\":\"dave\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.netexpertise.eu\/en\/#\/schema\/person\/image\/\",\"url\":\"http:\/\/1.gravatar.com\/avatar\/1129916e1f4955bd632f27f836f64e55?s=96&d=mm&r=g\",\"contentUrl\":\"http:\/\/1.gravatar.com\/avatar\/1129916e1f4955bd632f27f836f64e55?s=96&d=mm&r=g\",\"caption\":\"dave\"},\"sameAs\":[\"http:\/\/www.netexpertise.eu\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Netexpertise - MySQL \/ PostgreSQL on iSCSI Fail to Start at Boot","description":"Mysql and PostgreSQL on iSCSI disks fail to start at boot. Have you checked dependencies and the order in which systemd services start?","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.netexpertise.eu\/en\/systems\/linux\/mysql-postgresql-on-iscsi-fail-to-start-at-boot.html","og_locale":"en_US","og_type":"article","og_title":"Netexpertise - MySQL \/ PostgreSQL on iSCSI Fail to Start at Boot","og_description":"Mysql and PostgreSQL on iSCSI disks fail to start at boot. Have you checked dependencies and the order in which systemd services start?","og_url":"https:\/\/www.netexpertise.eu\/en\/systems\/linux\/mysql-postgresql-on-iscsi-fail-to-start-at-boot.html","og_site_name":"Netexpertise","article_published_time":"2021-05-16T08:33:55+00:00","article_modified_time":"2021-05-16T13:48:24+00:00","author":"dave","twitter_card":"summary_large_image","twitter_creator":"@netexpertise","twitter_site":"@netexpertise","schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.netexpertise.eu\/en\/systems\/linux\/mysql-postgresql-on-iscsi-fail-to-start-at-boot.html","url":"https:\/\/www.netexpertise.eu\/en\/systems\/linux\/mysql-postgresql-on-iscsi-fail-to-start-at-boot.html","name":"Netexpertise - MySQL \/ PostgreSQL on iSCSI Fail to Start at Boot","isPartOf":{"@id":"https:\/\/www.netexpertise.eu\/en\/#website"},"datePublished":"2021-05-16T08:33:55+00:00","dateModified":"2021-05-16T13:48:24+00:00","author":{"@id":"https:\/\/www.netexpertise.eu\/en\/#\/schema\/person\/cb4cd666549d22e9070ec1cfc1a496fa"},"description":"Mysql and PostgreSQL on iSCSI disks fail to start at boot. Have you checked dependencies and the order in which systemd services start?","breadcrumb":{"@id":"https:\/\/www.netexpertise.eu\/en\/systems\/linux\/mysql-postgresql-on-iscsi-fail-to-start-at-boot.html#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.netexpertise.eu\/en\/systems\/linux\/mysql-postgresql-on-iscsi-fail-to-start-at-boot.html"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.netexpertise.eu\/en\/systems\/linux\/mysql-postgresql-on-iscsi-fail-to-start-at-boot.html#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.netexpertise.eu\/en"},{"@type":"ListItem","position":2,"name":"MySQL \/ PostgreSQL on iSCSI Fail to Start at Boot"}]},{"@type":"WebSite","@id":"https:\/\/www.netexpertise.eu\/en\/#website","url":"https:\/\/www.netexpertise.eu\/en\/","name":"Netexpertise","description":"Systems \/ Networks \/ DevOps","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.netexpertise.eu\/en\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.netexpertise.eu\/en\/#\/schema\/person\/cb4cd666549d22e9070ec1cfc1a496fa","name":"dave","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.netexpertise.eu\/en\/#\/schema\/person\/image\/","url":"http:\/\/1.gravatar.com\/avatar\/1129916e1f4955bd632f27f836f64e55?s=96&d=mm&r=g","contentUrl":"http:\/\/1.gravatar.com\/avatar\/1129916e1f4955bd632f27f836f64e55?s=96&d=mm&r=g","caption":"dave"},"sameAs":["http:\/\/www.netexpertise.eu"]}]}},"_links":{"self":[{"href":"http:\/\/www.netexpertise.eu\/en\/wp-json\/wp\/v2\/posts\/1229"}],"collection":[{"href":"http:\/\/www.netexpertise.eu\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.netexpertise.eu\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.netexpertise.eu\/en\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/www.netexpertise.eu\/en\/wp-json\/wp\/v2\/comments?post=1229"}],"version-history":[{"count":0,"href":"http:\/\/www.netexpertise.eu\/en\/wp-json\/wp\/v2\/posts\/1229\/revisions"}],"wp:attachment":[{"href":"http:\/\/www.netexpertise.eu\/en\/wp-json\/wp\/v2\/media?parent=1229"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.netexpertise.eu\/en\/wp-json\/wp\/v2\/categories?post=1229"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.netexpertise.eu\/en\/wp-json\/wp\/v2\/tags?post=1229"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}