One of the secret powers of relational database systems is that there is no direct relationship between the query that describes the data you want and the execution path to retrieve it. The database is free to find optimizations and alternative paths to fetch data. Combined with database configurations such as indexes and partitions, you can frequently retrieve large amounts of data and run arbitrary queries on it without worrying too much about performance. when writing a query (We spend a lot of time thinking about performance when managing indexes and statistics collection).
That doesn’t mean there aren’t some…unique choices as to how to organize data “for performance”. Chris I needed to add a field to one of my data models. Since this data model was generated from an object-oriented mapping, it seemed easy to implement. Simply add fields to your object-oriented model, generate migration scripts, and start the rollout process.
In fact, it wasn’t easy.Because it was like this at first anything What we were doing with our basic configuration meant that the ORM tool we were using was not able to successfully generate the migration scripts. Ok, Chris was able to generate it manually. After all, it was he in one field. However, when we applied the changes to our test database, we realized that simply adding fields was not enough. Fetching fields required altering a number of materialized views. Chris also realized that simply changing the view wasn’t enough because when he opened the view’s code, the view didn’t work as expected.
Each view queried hundreds of tables, all with the same schema.They were all named with patterns MyDataset10OCT2023
. Each had a foreign key going back to the previous day’s table. MyDataset10OCT2023
link back to MyDataset09OCT2023
returned to MyDataset08OCT2023
, and so on.opinions combined all These tables together aggregate data across history, essentially summarizing a daily snapshot of how the data has changed.
All of this was managed by a set of PL/SQL stored procedures, some of which generated daily tables and some of which regenerated queries for materialized views. All of this means that Chris needs to add a field. each one daily dining table, or Generating the SQL query to replace the materialized view required manipulating PL/SQL code that concatenated strings.
Given the options, changing all the tables seemed easy and at least something I could automate and test.But Chris doesn’t know anything. why Basically, you have a linked list of database tables with daily snapshots, and if you ask anyone who’s been on the team for a while, you won’t get a better answer than “I think they did that for performance.” It was.