By default, postgres prefixes each log line with a timestamp. On NixOS
logs are written to journal anyway, so they include an external
timestamp, so the timestamp ends up being printed twice, which clutters
the log.
* Add a module option to change the log prefix.
* Set it to upstream default sans timestamp.
This seems to have worked in 15f105d41f (5
months ago) but broke somewhere in the meantime.
The current module doesn't seem to be underdocumented and might need a
serious refactor. It requires quite some hacks to get it to work (see
https://github.com/NixOS/nixpkgs/issues/86305#issuecomment-621129942),
or how the ldap.nix test used systemd.services.openldap.preStart and
made quite some assumptions on internals.
Mic92 agreed on being added as a maintainer for the module, as he uses
it a lot and can possibly fix eventual breakages. For the most basic
startup breakages, the remaining openldap.nix test might suffice.
slapd does only print the error and not the line number.
Sometimes it is not even clear that it fails to start
due to an incorrect configuration file.
Example output of slaptest:
5e1b2179 /nix/store/gbn2v319d4qgw851sg41mcmjm5dpn39i-slapd.conf: line 134 objectClass: Missing closing parenthesis before end of input
ObjectClassDescription = "(" whsp
numericoid whsp ; ObjectClass identifier
[ "NAME" qdescrs ]
[ "DESC" qdstring ]
[ "OBSOLETE" whsp ]
[ "SUP" oids ] ; Superior ObjectClasses
[ ( "ABSTRACT" / "STRUCTURAL" / "AUXILIARY" ) whsp ]
; default structural
[ "MUST" oids ] ; AttributeTypes
[ "MAY" oids ] ; AttributeTypes
whsp ")"
slaptest: bad configuration file!
A centralized list for these renames is not good because:
- It breaks disabledModules for modules that have a rename defined
- Adding/removing renames for a module means having to find them in the
central file
- Merge conflicts due to multiple people editing the central file
The redis module currently fails to start up, most likely due to running
a chown as non-root in preStart.
While at it, I hardcoded it to use systemd's StateDirectory and
DynamicUser to manage directory permissions, removed the unused
appendOnlyFilename option, and the pidFile option.
We properly tell redis now it's daemonized, and it'll use notify support
to signal readiness.
The default for logFile is /var/log/couchdb.log, and the tmpfile rules chown
${dirOf cfg.logFile}, which is just /var/log, to couchdb:couchdb.
This was found by Edes' report on IRC, which looked like
Detected unsafe path transition /var/log → /var/log/journal during canonicalization of /var/log/journal
While this bug has been present since the initial couchdb module in
62438c09f7 by @garbas, this wasn't a
problem, because the initial module only created and chowned /var/log
if it didn't exist yet, which can't occur because this gets created in
the initial phases of NixOS startup.
However with the recent move from manual preStart chown scripts to
systemd.tmpfiles.rules in 062efe018d (#59389),
this chown is suddenly running unconditionally at every system
activation, therefore triggering the above error.
This is rework of part of https://github.com/NixOS/nixpkgs/pull/46670.
My usecase was to be able to inspect PG datadir as wheel user.
PG11 now allows starting server with 0750 mask for data dir.
`groupAccess = true` now does this automatically. The only thing you have to do
is to set group ownership.
For PG10 and below, I've described a hack how this can be done. Before this PR
hack was impossible. The hack isn't ideal, because there is short
period of time when dir mode is 0700, so I didn't want to make it official.
Test/example is present too.
In commit d43dc68db3, @Mic92 split the
rootpw option to allow specifying it in a file kept outside the Nix
store, as an alternative to specifying the password directly in the
config.
Prior to that, rootpw's type was `str`, but in order to allow both
alternatives, it had to become `nullOr str` with a default of `null`. So
I can see why this assertion, that either rootpw or rootpwFile are
specified, makes sense to add here.
However, these options aren't used if the configDir option is set, so as
written this assertion breaks valid configurations, including the
configuration used by nixos/tests/ldap.nix.
So this patch fixes the assertion so that it doesn't fire if configDir
is set.
The change to "NixOS Test Cluster" in #59179 broke startup of existing clusters
that used the previously-default cluster name "Test Cluster":
ERROR 23:00:47 Fatal exception during initialization
org.apache.cassandra.exceptions.ConfigurationException: Saved cluster name Test Cluster != configured name NixOS Test Cluster
Fixes#63388.
Manual build broken by 79f7f89442, which
is part of pull request #59179 (Fix Cassandra, improve config and
tests).
The issue was just a small error because of an unbalanced <literal/>
tag, so only a "/" was missing :-)
Signed-off-by: aszlig <aszlig@nix.build>
Cc: @aanderse
If the `seedAddresses` is not set, don't force `SimpleSeedProvider` to
be in `seed_provider`. This could cause problems in a multi-datacenter
deployment when a different seed provider is preferred.
If you're on a multi user system you don't want to have the password in
the nix-store. With the new jmxRolesFile option you can specify your own
protected file instead.
mysql already has its socket path hardcoded to to
/run/mysqld/mysqld.sock.
There's not much value in making the pidDir configurable, which also
points to /run/mysqld by default.
We only seem to use `services.mysql.pidDir` in the wordpress startup
script, to wait for mysql to boot up, but we can also simply wait on the
(hardcoded) socket location too.
A much nicer way to accomplish that would be to properly describe a
dependency on mysqld.service. This however is not easily doable, due to
how the apache-httpd module was designed.
As we don't need to setup data directories from ExecStartPre= scripts
anymore, which required root, but use systemd.tmpfiles.rules instead,
everything can be run as just the mysql user.
define commands like "waiting for the mysql socket to appear" or "setup
initial databases" in a let expression, so the main control flow becomes
more readable.
We need to keep using `RuntimeDirectory=mysqld`, which translates to
`/run/mysqld`, as this is used for the location of the file socket, that
could differ with what is configured via `cfg.pidDir`.
This was a testing oversight that came from #61009 -- I forgot to test
the new traceFormat option with older server versions while I was
working on FDB 6.1.
Since trace_format is only available in 6.1+, emitting it
unconditionally caused older versions of the database fail to start,
reporting an error. We simply gate it behind a version check instead,
and assert the format is always XML on older versions. This avoids the
case where the user has an old version, changes traceFormat willingly,
and then is confused by why it didn't work.
As reported by @TimothyKlim in the comments on commit
c55b9236f0. See
c55b9236f0 (r33566132)
Signed-off-by: Austin Seipp <aseipp@pobox.com>
This is needed because some PostgreSQL plugins don't have a bin
directory. If only these plugins are listed in cfg.extraPlugins buildEnv
will turn $out/bin into a symbolic link to ${pg}/bin. Lateron we try to
rm $out/bin/{pg_config,postgres,pg_ctl} which will then fail because
$out/bin will be read-only.
which was wrongly specified as types.lines
Prevent it from getting copied to nix store as people might use it for
credentials, and make the tests cover it.
The default, which is /tmp, has a few issues associated with it:
One being that it makes it easy for users on the system to spoof a
PostgreSQL server if it's not running, causing applications to connect
to their provided sockets instead of just failing to connect.
Another one is that it makes sandboxing of PostgreSQL and other services
unnecessarily difficult. This is already the case if only PrivateTmp is
used in a systemd service, so in order for such a service to be able to
connect to PostgreSQL, a bind mount needs to be done from /tmp to some
other path, so the service can access it. This pretty much defeats the
whole purpose of PrivateTmp.
We regularily run into issues with this in the past already (one example
would be https://github.com/NixOS/nixpkgs/pull/24317) and with the new
systemd-confinement mode upcoming in
https://github.com/NixOS/nixpkgs/pull/57519, it makes it even more
tedious to sandbox services.
I've tested this change against all the postgresql NixOS VM tests and
they still succeed and I also grepped through the source tree to replace
other occasions where we might have /tmp hardcoded. Luckily there were
very few occasions.
Signed-off-by: aszlig <aszlig@nix.build>
Cc: @ocharles, @thoughtpolice, @danbst
The default galera_new_cluster script tries to set this environment
variable using systemctl set-environment which doesn't work if the
variable is not being used in the unit file ;)
Although the package itself builds fine, the module fails because it
tries to log into a non-existant file in `/var/log` which breaks the
service. Patching to default config to log to stdout by default fixes
the issue. Additionally this is the better solution as NixOS heavily
relies on systemd (and thus journald) for logging.
Also, the runtime relies on `/etc/localtime` to start, as it's not
required by the module system we set UTC as sensitive default when using
the module.
To ensure that the service's basic functionality is available, a simple
NixOS test has been added.
As cassandra start script hardcodes the location of logback
configuration to `CASSANDRA_CONF_DIR/logback.xml` there is no way to
pass an alternate file via `$JVM_OPTS` for example.
Also, without logback configuration DEBUG level is used which is not
necessary for standard usage.
With this commit a default logback configuration is set with log level
INFO.
Configuration borrowed from:
https://docs.datastax.com/en/cassandra/3.0/cassandra/configuration/configLoggingLevels.html
This cleans up the CockroachDB expression, with a few suggestions from
@aszlig.
However, it brought up the note of using systemd's StateDirectory=
directive, which is a nice feature for managing long-term data files,
especially for UID/GID assigned services. However, it can only manage
directories under /var/lib (for global services), so it has to introduce
a special path to make use of it at all in the case someone wants a path
at a different root.
While the dataDir directive at the NixOS level is _occasionally_ useful,
I've gone ahead and removed it for now, as this expression is so new,
and it makes the expression cleaner, while other kinks can be worked out
and people can test drive it.
CockroachDB's dataDir directive, instead, has been replaced with
systemd's StateDirectory management to place the data under
/var/lib/cockroachdb for all uses.
There's an included RequiresMountsFor= clause like usual though, so if
people want dependencies for any kind of mounted device at boot
time/before database startup, it's easy to specify using their own
mount/filesystems clause.
This can also be reverted if necessary, but, we can see if anyone ever
actually wants that later on before doing it -- it's a backwards
compatible change, anyway.
Signed-off-by: Austin Seipp <aseipp@pobox.com>
This also includes a full end-to-end CockroachDB clustering test to
ensure everything basically works. However, this test is not currently
enabled by default, though it can be run manually. See the included
comments in the test for more information.
Closes#51306. Closes#38665.
Co-authored-by: Austin Seipp <aseipp@pobox.com>
Signed-off-by: Austin Seipp <aseipp@pobox.com>
This allows, finally, proper detection when postgresql is ready to
accept connections. Until now, it was possible that services depending
on postgresql would fail in a race condition trying to connect
to postgresql.
While this seems silly at first (it's already given as start parameter
to mysqld), it seems like xtrabackup needs that sometimes.
Without it, a Galera cluster cannot be run using the xtrabackup
replication method.
