Tags: Summary Conclusions Recommendations Research PaperHow To Solve Applied Mathematics ProblemsHow Should I Start My Scholarship EssayScience Research Paper FormatTerm Paper Price DiscriminationFree DissertationsAbd All But DissertationProblem Solving In C
The paper Optimizing Space Amplification in Rocks DB suggests that this can be fixed by changing the level sizes so that instead of insisting that L3 has exactly 1000 sstables, we focus on L3 having 10 times more sstables than L2.Neither Scylla nor Cassandra have this fix yet, so in worst case during massive overwrites, their LCS may still have space amplification of 2.So at most, we can have 10% duplicated data (if all the data in L1 and L2 happens to be overwrites to data that we have in L3).
E.g., consider that we have a filled L2 with 100 sstables but L3 also has just 100 sstables (and not 1000).
In this case, the last level only has about half of the data, half of the data may be duplicated, so we may see 2-fold space amplification.
The first thing that Leveled Compaction does is to replace large sstables, the staple of STCS, by “runs” of fixed-sized (by default, 160 MB) sstables.
A run is a log-structured-merge (LSM) term for a large sorted file split into several smaller files.
This post and the rest of this series are based on a talk that I gave (with Raphael Carvalho) in the last annual Scylla Summit in San Francisco.
The video and slides for the talk are available on our Tech Talk page.
As unfortunate this is, it is of course not nearly as bad as the 8-fold space amplification we saw for STCS.
In the previous post, we looked at two simple examples to demonstrate STCS’s high space amplification. The first example was straightforward writing of new data at a constant pace, and we saw high temporary disk space use during compaction – at some points doubling the amount of disk space needed.
In other words, a run is a collection of sstables with non-overlapping token ranges.
The benefit of using a run of fragments (small sstables) instead of one huge sstable is that with a run, we can compact only parts of the huge sstable instead of all of it.