This module provides persistency for rdf_db.pl based on the
rdf_monitor/2 predicate to track changes to the repository. Where
previous versions used autosafe of the whole database using the
quick-load format of rdf_db, this version is based on a quick-load file
per source (4th argument of rdf/4), and journalling for edit operations.
The result is safe, avoids frequent small changes to large files which
makes synchronisation and backup expensive and avoids long disruption of
the server doing the autosafe. Only loading large files disrupts service
for some time.
The persistent backup of the database is realised in a directory, using
a lock file to avoid corruption due to concurrent access. Each source is
represented by two files, the latest snapshot and a journal. The state
is restored by loading the snapshot and replaying the journal. The
predicate rdf_flush_journals/1 can be used to create fresh snapshots and
delete the journals.
- See also
- - rdf_edit.pl
- To be done
- - If there is a complete `.new' snapshot and no journal, we should
move the .new to the plain snapshot name as a means of recovery.
- - Backup of each graph using one or two files is very costly if there
are many graphs. Although the currently used subdirectories avoid
hitting OS limits early, this is still not ideal. Probably we
should collect (small, older?) files and combine them into a single
quick load file. We could call this (similar to GIT) a `pack'.
- rdf_attach_db(+Directory, +Options) is det
- Start persistent operations using Directory as place to store
files. There are several cases:
- Empty DB, existing directory
Load the DB from the existing directory
- Full DB, empty directory
Create snapshots for all sources in directory
- One of
read_only. Read-only access implies that the RDF
store is not locked. It is read at startup and all
modifications to the data are temporary. The default
auto mode is
read_write if the directory is
writeable and the lock can be acquired. Otherwise
it reverts to
- Number of threads to use for loading the initial
database. If not provided it is the number of CPUs
as optained from the flag
- Maximum number of journals kept open. If not provided,
the default is 10. See limit_fd_pool/0.
- Number of levels of intermediate directories for storing
the graph files. Default is 2.
false), do not print informational
messages. Finally, if
brief it will show minimal
true, nested log transactions are added to the
journal information. By default (
false), no log-term
is added for nested transactions.\\
permission_error(write, directory, Directory)
- rdf_persistency_property(?Property) is nondet
- True when Property is a property of the current persistent database.
Exposes the properties that can be passed as options to
rdf_persistency_property(access(read_only)) is true iff the database
is mounted in read-only mode. In addition, the following property is
- The directory in which the database resides.
- rdf_detach_db is det
- Detach from the current database. Succeeds silently if no
database is attached. Normally called at the end of the program
- True if Dir is the current RDF persistent database.
- Flush dirty journals. Options:
- Only flush if journal is over KB in size.
- Only flush the journal of Graph
- To be done
- - Provide a default for min_size?
- rdf_persistency(+DB, Bool)
- Specify whether a database is persistent. Switching to
kills the persistent state. Switching to
true creates it.
- rdf_db:property_of_graph(?Property, +Graph) is nondet[multifile]
- Extend rdf_graph_property/2 with new properties.
- rdf_journal_file(+Graph, -File) is semidet
- rdf_journal_file(-Graph, -File) is nondet
- True if File the name of the existing journal file for Graph.
- rdf_snapshot_file(+Graph, -File) is semidet
- rdf_snapshot_file(-Graph, -File) is nondet
- True if File the name of the existing snapshot file for Graph.
- rdf_db_to_file(+DB, -File) is det
- rdf_db_to_file(-DB, +File) is det
- Translate between database encoding (often an file or URL) and
the name we store in the directory. We keep a cache for two
reasons. Speed, but much more important is that the mapping of
raw --> encoded provided by www_form_encode/2 is not guaranteed
to be unique by the W3C standards.