Katmai, the code name for Microsoft's imminent SQL Server 2008 release, comes from an Alaskan territory know for volcanoes, which may not be the best symbol for a database. So far, however, Katmai hasn't blown up on me. And the lower-profile Katmai seems like a good follow-on to Yukon, the code name for the gigantic SQL Server 2005 release.
Building on the sweeping, enterprise-oriented improvements in SQL Server 2005 (see review), Katmai sports very nice new features for large deployments. Among the more touted attributes in the database engine are data and backup compression, sparse columns, and compressed and filtered indexes, all of which are geared to saving storage space, as well as Change Data Capture, which captures changes to production data in tables that can be used to update a data warehouse.
[ Compare Katmai to Oracle Database 11g. See "Lab test: Oracle Database 11g shoots the moon." ]
These are just the tip of the iceberg, or volcano, and there are of course many other new features – such as policy-based management – that will appeal to large and small shops alike. Every aspect of the product has been touched significantly.
More Data, Less Storage
For starters, there are two types of data compression: row and page. They do, in fact, compress data in different ways, so it's important to understand the benefits of each, as well as how they work. Row compression is true compression, whereby the engine removes unused spaces at the ends of columns and, thus, saves space. This is the same technique SQL Server already uses for vardecimal compression; Microsoft has just expanded the use to other data types.
Page compression does what's known as dictionary compression, in that it normalizes the data on each page and keeps a lookup pointer. This is essentially the same trick used in Oracle Database 11g, which Oracle calls Oracle Advanced Compression. Without getting too much into the pros and cons of each, it's worth noting that SQL Server's page compression includes the lower-level row compression. In other words, if you have page compression turned on, you automatically get row compression.
Microsoft has included a couple of stored procedures to help you estimate both the level of savings you'll get with each method before you compress, and how much expansion will result if you uncompress the database later. This is an important and really thoughtful feature because you need to know not only if compression will be worth your time, but also if your disk can handle the uncompressed data should you need to revert. Just keep in mind that the procedures work on a small yet statistically significant random sampling of the data. You could get some bad estimates if the query happens to hit a poor representation of your data.
Plus, the way Microsoft implements compression spares more than storage resources. The data stays compressed in memory and only gets decompressed when read, meaning that you can fit more data pages into memory. This should save disk fetches, and the CPU it takes to decompress will be far less expensive than the disk seek would have been.
The sparse columns feature allows you to store null values without taking up any physical space. If you have a large table with a lot of null values in a column, you can waste ample disk space keeping track of those nulls. Storing nulls in sparse columns takes zero space, so your storage requirement goes way down.
One big caveat with sparse columns is that they don't work with compression. Frankly, this is a big mistake by Microsoft, and I hope the company does right by its users, pushing this fix into a service pack instead of waiting for the next release. In the meantime, if you have sparse columns defined on a table, don't expect to compress your data on it as well. I honestly don't know what Microsoft was thinking with this one, but this should never have gotten out the door. Sparse columns and compression are a perfect match; this one may deserve a Darwin Award.
Compressed indexes are just what they sound like: another opportunity to save storage space. Filtered indexes allow you to put a where clause (just like a query) on your index so that only a portion of your table is indexed. It may seem counterproductive, but there are several instances where you would want to filter an index. The perfect example is with sparse columns. Instead of keeping an index that contains mostly nulls, you would put an index on the sparse column where the value does not equal null. This way, only the rows with actual values will be indexed and the size of your index decreases significantly.