Updated my notes.

README

@@ -36,6 +36,9 @@ these reports, and we only have so much free time: the team itself is the
 weak point in the operating chain. It's a good idea, then, to automate
 as much as possible, and so reduce the bottleneck.
 
+While that's being done, I might as well also make some improvements to
+other areas of BOSS.
+
 
 LOOT Design Overview
 ====================
@@ -200,14 +203,6 @@ A general workflow for a user running LOOT to sort would be:
 4. LOOT then applies the resulting load order, saves any user choices
    as userlist rules, and exits.
 
-Because auto-sorting will require significant computation (think CBash
-Bashed Patch building), optimisations will be made where possible. For
-example, if LOOT is run and the plugins have not changed since it was
-last run, then it will use the last load order it calculated. It will
-also do this if the only change was removal of plugin(s). Addition of
-plugins will require a re-calculation, but it may be possible that only
-a partial scan would be required.
-
 LOOT must also provide a means to manage user rules, to undo its
 changes (only back one run though), and to edit its settings.
 
@@ -216,6 +211,11 @@ The Userlist Syntax and Masterlist Syntax docs will be merged into a
 will be structured so that there is a progression in complexity from
 simple rule syntax to complex rule syntax as the guide progresses.
 
+The drag 'n' drop plugin positioning will likely require userlist rules
+to support positioning of plugins precisely relative to other plugins, but it
+may be possible to achieve the positioning using only the priority setting. If
+possible, that is preferred.
+
 
 Implementation
 ==============
@@ -240,9 +240,42 @@ relocate their hosting, the locations of the online masterlists will
 not be hardcoded, but instead specified in LOOT's ini file. More generally,
 hardcoding of potential variables will be avoided where possible.
 
+Because auto-sorting requires LOOT to know what is inside the plugins it sorts,
+it must scan the plugins, and determine which of its records are new and which
+are edited (new records aren't used for determining load order). However, this
+requires significant computation, so LOOT will use a database that holds which
+records each plugin changes. The database will be hosted online, and will be
+used in the following manner:
 
-LOOT Database Format
-====================
+  1. LOOT checks to see if a database update is available, and downloads the
+     update if so. This may be done by diff or by downloading the whole file if
+     a diff is not feasible.
+  2. LOOT scans the installed plugins, calculating the CRC of each plugin.
+  3. For each plugin, the database is scanned for an entry with a matching file
+     name and CRC. If an entry is found, the entry's data is used and scanning
+     of the plugin is skipped.
+  4. Any plugins which do not already have database entries are scanned.
+  5. The correct load order of the plugins is calculated using data from scanned
+     plugins, the database, the masterlist and the userlist.
+  6. The data from the scanned plugins is appended to the database file, which
+     is then uploaded to its hosted location.
+     
+In addition to having this database file as a sort of permanent and 
+self-updating cache, optimisations can also be made to skip load order 
+calculation. For example, if the only changes from the last load order set by
+LOOT is that a plugin or plugins have been removed, then the last load order
+may be used. It might also be possible to only perform partial calculation
+when plugins are added, though this is not certain.
+
+Note: For clarity, the database that holds the results of plugin scanning will
+be referred to as the "machine database" (or MDB) as it will only be read and
+written by the LOOT code. The list of metadata that is maintained by the LOOT
+team will be referred to as the "masterlist", and the list of metadata that is
+individual to each user will be referred to as the "userlist".
+
+
+LOOT Masterlist Format
+======================
 
 Simple Example
 --------------
@@ -339,3 +372,40 @@ globals:
     - <<: {*OBSOLETE, content: Quotes aren't necessary, unless the message contains special chars. }
 ...
 }}}
+
+
+LOOT Machine Database Format
+============================
+
+The formatting of this database has not yet been decided. It doesn't need to be
+human-readable, but must contain the following information for each plugin:
+
+  * Filename
+  * CRC
+  * Masters in the order they appear in the plugin's master list.
+  * FormIDs of the records it edits (not added records).
+  
+Filename can be a null-terminated byte string, as can the master filenames. The
+CRC is a 32 bit unsigned integer, as are the FormIDs. To aid searching of the 
+database, it might be worth including the filename byte length as an 8 bit
+unsigned integer preceding the filename, and including the total byte length of
+the master and FormID lists as a 32 bit unsigned integer following the CRC. 
+This would allow the search to skip the bulk of the database file.
+
+So, one database entry might be as follows:
+
+{{{
+Filename length			uint8_t						Length includes null character.
+Filename				char[filename length + 1]	Final char is NULL.
+CRC						uint32_t
+Data size				uint32_t					Length of the Masters
+													+ FormIDs sections.
+Masters length			uint32_t					Length of the Masters section.
+													Includes null characters.
+Masters					char *						A list of null-terminated
+													filenames.
+FormIDs					uint32_t[FormIDs number]
+}}}
+
+It might be worth indexing the filename and CRCs, and storing offsets to the
+data for each plugin.