jcoffey-dev is traveling from Thursday 1 October through Sunday 4 October. Issues and pull requests are welcome, and will get an answer after that. Thanks for your patience.
Phase 2 of the personal-data catalog spec.
resources/privacy/catalog.toml classifies every object in the schema
(316) and inbuxa's own JMAP objects (12): each property that can hold
personal data, with its categories, and for objects that hold any,
whose data it is, where it lives, its scope and what bounds its
retention (a named setting where there is one). Twenty sources that
are no object -- the log file, exporters, webhooks, spam lookups, the
Explain cache, relays and hooks, push, legacy-use records -- carry the
same facts plus the settings that turn them on, whether the data
leaves the host, and the code that writes it. Classifications of
objects that hold data about people are from the spec's source map;
the rest are typed from the schema alone (address, IP, secret).
tools/fork/privacy-check.py fails CI when an object or inbuxa object
has no entry, when a property the schema types as an address, IP or
secret is left to its object's default, when an entry names an
object, property, setting or code path that is gone, or when it uses
a word outside the catalog's vocabulary. --unlisted prints starting
entries. strip.py's report gains "Unclassified in the privacy
catalog": objects and fields new in an import and not classified,
informational like the Enterprise flags.
Tested: 13 unit tests (tools/fork/tests): the check passes on this
tree; fails on an unclassified object, an address hidden behind a
default, a secret in a set or object reference, stale properties,
objects, settings and code paths, an unlisted inbuxa object and a
word outside the vocabulary; --unlisted's entries; and the strip
report on a synthetic import. The check and the tests run in the
fork-checks job.
strip.py compiles the stripped tree, so a dual-licensed file that only
serves an Enterprise feature fails the import instead of the merge, as
v0.16.23's tests/src/directory/issuer.rs does. Upstream's tests of the
features the fork rebuilt are expected not to compile there and are listed
in build-check-known.txt; an error anywhere else fails the run. Checked
against both imports: v0.16.22 passes with its 16 expected errors, v0.16.23
fails on issuer.rs alone. Imports the strip leaves unused are reported.
It also renames the upstream name where clients, users or operators meet
it as an identifier, from tools/fork/renames.py: wire-protocol names, the
web interface's client id, store keys, configuration defaults and the
served schema. main is renamed with the same module, so a re-import
arrives purged and those lines don't conflict.
notice-check.py fails CI when an upstream file the fork changed, measured
against the upstream branch, lacks its AGPL 5(a) notice; --fix adds it.
It runs beside the name check in a renamed fork-checks job.
Also commits v0.16.23's strip report under docs/fork/strip-reports/, which
the import in #18 left out.
A few of upstream's dual-licensed files carry code from other projects
under MIT or BSD terms. The fork redistributes it, so their licenses
require the notices to travel with it. THIRD-PARTY.md reproduces them.
strip.py now reads the stripped tree's comments for another copyright
holder, another license, or a note that code came from somewhere else, and
names any file THIRD-PARTY.md doesn't cover. It reports, never fails: the
notice goes in with the merge that brings the release in.
On v0.16.22 it finds 14 files, all of them covered. The rest of the report
is byte-for-byte what the committed one says, so the scan disturbs nothing
it already did.
Upstream's Dockerfiles and CI pass --features "... enterprise" on the cargo
command line, which the manifest edits don't reach. 11 occurrences at
v0.16.22. Dockerfile.build and Dockerfile.fdb are now scanned too, whatever
their extension.
jcoffey-dev