Use Case #16: SKA Global Data Lake PoC for Astronomy-scale Data Products
Metadata Performance tests
We want to know how performant the Rucio metadata is and know limitations
How many key/values to test? 100/1000? Measure time to retrieve a list of files based on x number of key/value pairs?
Theoretical exercise for SKA SRC Scale
Outcome - description of how JSON vs column metadata should be used, description of potential use of nested metadata to create "namespace" behaviour, description of scaling for astronomy metadata and SKA archive scale.