Yesterday I discussed how I'd build a codeforge for & upon our string-centric hardware (as well as clientside feedreader, crash reporter, & fallback crash reporter) comprising of a repo viewer & issue tracker. What else would we want to include in it?
We'd want to let you down the AutoMerge documents to fork them & attach them to issues to request they be merged. We may add a minor integration between the 2 components to merge with a click of a button, but that might not be worth it.
This catalog would list all repos, issue tracker(s), mailing lists, chatrooms, websites, etc for each subproject. Though I wouldn't want to mandate that subprojects use our services. The repos may be processed to extract the permissions each component requires.
This catalogue would be useful for performing analysis over the entire project!
We can index the input formats each program supports, so our devices can download it & offer to install software for new file formats it encounters.
After parsing commandline flags (I see internationalization but not its initialization) validating (in some cases, single) args remain elfcompress parses each remaining arg aggregating error codes, before tidying up!
For each it opens the given ELF file validating its kind & fstat(), retrieves the E, S, & P headers, considers calculating a last-offset based on that, & iterates over the ELF sections.
For each section (round 2) it retrieves various properties upon successfully retrieving the section whilst applying any requested compression, creates a new section in the output ELF file, & copies the various data over to it with minor tweaks (and more compression!).
It finishes by possibly finalizing various global names possibly adding compression, retrieves the updated E header, retrieves S header string index, possibly iterates over those names, updates layout info & tidies up!
For each section name elfcompress retrieves the index & S header, possibly computes some layout info, writes the S header, & checks if we're at the desired entry. If we have it retrieves various layout info & possibly adds compression!
There's a handful of helper wrapper functions dealing with additional details of how ELF files are laid out.
3.5/3.5! Fin for today! I don't want to get started on another command at this point...
We could get far on our hypothetical string-centric hardware collaborating peer-to-peer using the CRDT version control system I described the other day! But giving the project a central server (running on our custom hardware, lets say) can streamline development whilst adding quality control, so how'd we implement it?
We'll implement our "codeforge" as a suite of independent services, using an (as yet, proprietary) <form> control for authentication.
Perhaps the other most-important feature of a codeforge is an issue tracker. For this I'd start from a webframework which uses the Parsing Unit for URL routing & data lookups, the Output Unit for templating, & the Arithmetic for less-trivial queries. Allowing webforms to update the stored labelled-tree of data.
Once we've datamodelled "issues" we can render them to HTML & webfeeds, & provide forms to create nwe issues & comments thereupon.
We may include code to render issues to/from emails, though the previously-mentioned authentication widget could allow anyone to use the web UI.
Furthermore I'd want to have the OS automatically pull up this form in response to any crashes. But what if the form engine itself crashes? Then I'd task the BIOS team to build a fallback interface, offering a simpler (UDP?) endpoint for it to call. Akin to the fuzzer, I'd simplify/anonymize the crash report.
@alcinnz What a coincidence! I was just thinking the same thing. If I could choose a life without technology around forever, I would, but I can't stand a short break from it. FOMO is a real struggle. 🙃
Diffing can be incredibly helpful for managing software! It can be used to retrofit CRDT support, monitor changes being made, guide how the software should be adjusted to match desired output(s), etc. Maybe for our Output Unit we'll have tool which autoruns & diffs a converter to guide its development, maybe adding automation where we find it worth the effort?
So how'd we implementing diffing on our string-centric hardware? Strings specifically? Its not trivial!
If the document's already stored as a CRDT we could just ask which edits are in one CRDT-document vs the other. Maybe postprocessing it to determine when both peers made equivalent edits!
As for state snapshots...
The core (Dynamic Programming) algorithm for a optimal text diff involves a building table of the max number of common characters between 2 strings. Common chars increment this count, differing chooses an optimal path. Then traverse a path back-to-front to output.
The increments & compares are well within the capabilities of even our Arithmetic Core. Though being a 16bit machine we may want to operate in UTF-16, or a custom variant with a larger character-space.
The catch is for strings a & b long this takes on the order a*b bytes. So if we're diffing 2 1kb files (not that big...) we'd need an entire mb! Whereas I've established that this Arithmetic Core has 64kb at most (in its slower Harvard mode).
We have "Dynamic Programming" (named to sound impressive to the researchers' generals) which resembles "divide & conquer" algorithms, but we store the partial solutions so we can relate them in a non-obvious order.
These strategies describe most of the popular algorithms we see all over computing & computer science!
But where all the real ingenuity lies is in designing datastructures, which can require mindset changes!
Yesterday I discussed collaboratively editing text, but there's other things you might want to edit!
Implementation notes: I'd keep the data in RAM in its sorted & compressed form, only decompressing it transiently for processing & modifying. Zipping from columnar format into "edit" rows would be most involved step.
For instance we'd want to edit not only individual files but whole directories of them! And the attributes attached to each of its files!
To implement such a "mapping" CRDT we can define an operation which sets the value at a given key to a given value, possibly to a "tombstone" value to indicate deletions. The problem comes when 2 peers attempt to set the same key to different values!
If we track our understanding of the latest edits our peers are aware of (a "lampart timestamp"), storing that in our edits' metadata, we can establish a causal ordering to these edits. When that fails we can compare peer IDs.
This causal sort is also vital for ordering text insertions!
Talking about text... Lists would be implemented basically the same way but with greater freedom regarding which data you can store in each node.
We may want to incorporate counters, by summing all the nudges up/down collaborators have made.
And once we have collections we'd want to store numbers, constant strings, booleans, etc in them. Combine these datatypes and we should be able to define most other CRDTs you might want.
Once again wishing the hypothetical device I've long been describing was real!
An OS & hardware focused specifically on the task of communicating with diverse people! Using existing protocols. Barely any apps!
Just today I introduced an editor framework to it so basically every editor has infinite undo, collaboration, & composability! (I wasn't responding to anything, just how the scheduling worked out...)
This a bit more "from scratch" than I'm capable of though...
Seeing all the encouragement to try "Linux" & the common pushbacks against it...
If you insist your existing apps should run on "Linux"... I'm impressed by how imperfectly well we've achieved this! Its significantly easier said then done.
To me its very much akin to saying "Don't blame people for driving when our infrastructure is built for cars". There too I'd say we shouldn't be driving.
Continuing my study of ELF Utils' commandline tools...
After initialization both I/O & internationalization as well as parsing commandline flags ar configures LibELF to a specified format version, parses/validates the commandline flags some more ensuring additional args remain, pops the archive name as a commandline arg, & branches over the subcommand specified by those flags.
This may output some help text via LibArgP. Or it may...
For Insert operations ar opens the ELF file, initializes some memory including a symboltable, for oper_qappend operations populates a hashmap ( LibC's search.h one), iterates over the ELF file, followed by extensive validation & cleaning up! For each ELF header (skipping filepaths for performance's sake) initializes a new archive entry for it, considers whether its present in the hashmap, & prepends the entry into a linkedlist.
Primarily ar performs "extractions" ("list" & "print" are variations), involving opening the archive & populating a hashmap, fstat()s the file, iterates over ELF entries, possibly atomically overwrites the file with extracted symbol table, & cleans up. For each entry it handles filepaths specially, extracts it into a new ELF symboltable, and/or checks for presence in the hashmap, & possibly formats human output with lots of options.
@alcinnz
I had limited success with elementary installs for previous Mac users... Nowadays Linux mint is my goto for people who just want something that'll let them use their computer.