Design Notes

• When POST-ing, messages to read-only and unidirectional groups are denied.
• Rudimentary mail checking is also done as POST time by looking up A and MX records for the domain in question.
• Reticule consists of four processes that communicate via Unix-domain sockets.
  • The first implements NNTP, and is used by clients when reading messages. It does no storage management.
  • The second is the storage manager. It basically sits waiting for new messages from its upstream, and accepts these. The messages it gets are tagged with group name and article number, and it just accept these, puts the message where indicated, and updated the active and history files, and pumps out the new info to the NNTP process.
  • The third is only run on the main Reticule node. It gets a message from the mail-to-news script, sees where it's supposed to go, assigns it an article number, and feeds it to the second process. It also does header mangling and all that kind of stuff. This is the "injector".
  • The fourth does streaming from the main Reticule node to all the other nodes. It's also only run on the main node. This is the "streamer".
• If the injector receives a message with a Message-ID it's already seen, it adds an Xrefs header that includes the previously seen instances. It then stores the message, as usual, in the group in question. This means that all the messages with the same Message-ID will basically be somewhat different -- they will have different Received headers, for instance, and may have different trailers and stuff.

  After it has stored this message, it then updates the Xrefs headers on all the other instances. This means that if message X first arrives in group A, it's handled as normal, and fed out to the nodes. Then message X arrives in group B, and it's handled, fed out, and then the instance in group A is altered, added to the history file again, and fed out again.

• The history file is a one-line-per-message file sorted in arrival order. It has five components -- a sequence number, a time stamp, an expiry time stamp, the Message-ID, and the group/article spec for one of the instances.
• When other nodes want a feed from the main node, they just ask the main node "give me all articles since sequence number XXXX".
• When an article is altered -- retroactively cross-posted, or encrypted or decrypted, or canceled -- it's added to the history file once again, which will mean that it's fed out to the other nodes again, which means that when things change on the main site, they'll change momentarily on all the other nodes, keeping things in sync globally.
• There may be different virtual servers on the same NNTP server. For instance, a filter excluding all spam messages may be implemented as a virtual NNTP server.
• It would be nice to extend NNTP to allow people to report stuff as spam via the NNTP server, but that would require participation from the clients. Other nice stuff might include cooperative scoring and the like. Anyway, that's probably not what comes first.
• Perhaps the easiest way to implement the NNTP bit (without using capabilities to be able to bind port 119) is having a very simple C command runn off of inetd.conf, that would simply talk to the NNTP server via Unix-domain sockets. That would mean that nothing would have to be run as root, which would be a relief.
• I don't know whether it would make more sense to pass the POSTs back to the main node through mail or via some other protocol. By using mail, I don't have to write spooling and stuff like that, but it might be nice not to have to maintain that whole ...@moderator.gmane.org thing.
• Groups can be renamed -- selecting the old name will give you the new name.