Also includes the ability to list buckets and objects in buckets from the ref store, which are not very efficent and mostly meant to be used for background tasks like this.
#rb carl.bystrom
[CL 28082400 by Joakim Lindqvist in ue5-main branch]
This also allows us set the default keyspace from the connection string for connections to scylla and cassandra, if you were not specifying a connection string nothing changes but if you are you will need to also specify the default keyspace.
We also introduce a gc setting to control the default gc policy used if not set in a namespace policy.
[CL 27067566 by Joakim Lindqvist in ue5-main branch]
This breaks field filtering on json objects because system.text.json does not yet support solving that but .NET 7 adds support for it.
#preflight none
[CL 25143384 by Joakim Lindqvist in ue5-main branch]
There are two parts to this, first we create a blob_index_v2 which does not include references and were we use a clustering key for regions. This allows us to have a much leaner row for this very frequently requested data.
Secondly, and the more impactful, is moving the references into a seperate table were we can use clustering key based on the hash of the refering object which means we no longer need a collection of references which should give much better performance from scylla.
This also includes the foundation for tracking references between blobs but this is not used for the ddc api.
With this we also remove the need for different table layouts between Scylla and base Cassandra.
We will automatically look for old data and migrate it if the setting MigrateFromOldBlobIndex is set (which it is by default).
#preflight none
[CL 24280396 by Joakim Lindqvist in ue5-main branch]
Cleaning all namespaces at the same time, reduces load on database.
Move last access tracking in scylla to a seperate table, reduces work on database for compaction due to large inlined blobs.
Also removed the deletion of blobs when GCing ref, as this only runs in a single region so it makes regions very inconsistent in how quickly they delete blobs.
Also added option to control if replication log gets a delete record, and disabled this by default, as this delete record is not used for anything and just adds noise to the replication log.
Added test for scylla shard scanning of ref GC to make sure it isn't broken.
#preflight none
[CL 24119154 by Joakim Lindqvist in ue5-main branch]
