When working with InnoDB, you have two ways for managing the tablespace storage:
- Throw everything in one big file (optionally split).
- Have one file per table.
I will discuss the advantages and disadvantages of the two options, and will strive to convince that innodb_file_per_table is preferable.
A single tablespace
Having everything in one big file means all tables and indexes, from all schemes, are ‘mixed’ together in that file.
This allows for the following nice property: free space can be shared between different tables and different schemes. Thus, if I purge many rows from my log table, the now unused space can be occupied by new rows of any other table.
This same nice property also translates to a not so nice one: data can be greatly fragmented across the tablespace.
An annoying property of InnoDB’s tablespaces is that they never shrink. So after purging those rows from the log table, the tablespace file (usually ibdata1) still keeps the same storage. It does not release storage to the file system.
I’ve seen more than once how certain tables are left unwatched, growing until disk space reaches 90% and SMS notifications start beeping all around.
There’s little to do in this case. Well, one can always purge the rows. Sure, the space would be reused by InnoDB. But having a file which consumes some 80-90% of disk space is a performance catastrophe. It means the disk needle needs to move large distances. Overall disk performance runs very low.
The best way to solve this is to setup a new slave (after purging of the rows), and dump the data into that slave.
InnoDB Hot Backup
The funny thing is, the ibbackup utility will copy the tablespace file as it is. If it was 120GB, of which only 30GB are used, you still get a 120GB backed up and restored.
mysqldump, mk-parallel-dump
mysqldump would be your best choice if you only had the original machine to work with. Assuming you’re only using InnoDB, a dump with –single-transaction will do the job. Or you can utilize mk-parallel-dump to speed things up (depending on your dump method and accessibility needs, mind the locking).
innodb_file_per_table
With this parameter set, a .ibd file is created per table. What we get is this:
- Tablespace is not shared among different tables, and certainly not among different schemes.
- Each file is considered a tablespace of its own.
- Again, tablespace never reduces in size.
- It is possible to regain space per tablespace.
Wait. The last two seem conflicting, don’t they? Let’s explain.
In our log table example, we purge many rows (up to 90GB of data is removed). The .ibd file does not shrink. But we can do:
ALTER TABLE log ENGINE=InnoDB
What will happen is that a new, temporary file is created, into which the table is rebuilt. Only existing data is added to the new table. Once comlete, the original table is removed, and the new table renamed as the original table.
Sure, this takes a long time, during which the table is completely locked: no writes and no reads allowed. But still – it allows us to regain disk space.
With the new InnoDB plugin, disk space is also regained when execuing a TRUNCATE TABLE log statement.
Fragmentation is not as bad as in a single tablespace: the data is limited within the boundaries of a smaller file.
Monitoring
One other nice thing about innodb_file_per_table is that it is possible to monitor table size on the file system level. You don’t need access to MySQL, to use SHOW TABLE STATUS or to query the INFORMATION_SCHEMA. You can just look up the top 10 largest files under your MySQL data directory (and subdirectories), and monitor their size. You can see which table grows fastest.
Backup
Last, it is not yet possible to backup single InnoDB tables by copying the .ibd files. But hopefully work will be done in this direction.
Hi,
They guy who absolutely hates innodb_file_per_table would be Domas Mituzas.
I disagree with part of his arguments. But he is an expert in the field, so this does not mean I belittle him. It’s just that there’s some contention between “what’s utterly best for performance” and “what’s best to live with”. The two do not work well together.
There’s reasons for and against. I’ve presented the reasons “for”.
As I see it, the development is moving in the direction of innodb_file_pre_table: the innodb plugin has to use this option if you want to use Barracuda table format (which allows for compression, among other things).
Anyway, I don’t get paid to support innodb_file_per_table 🙂 I’ll use the single tablespace if I think it’s superior.
Regards
I find the whole tone of Domas’ post extremely irritating. Internet trolls are a dime a dozen, but when someone really knows their stuff, it’s annoying when they go around with an attitude like anyone without that domain knowledge is an idiot. Especially when he makes hand-wavy assumptions about real world scenarios that aren’t universally true, such as the idea that permanent data will always grow faster overall than temporary data. Not to mention cases where there are other infrastructure considerations in play, such as using Time Machine on OS X where monolithic files defeat the primary utility of the system.
It sort of reminds me of Fabian Pascal’s dbdebunk.com mysterious silence coinciding very closely with the rise of the NoSQL movement. That guy was one of the foremost experts in relational theory, and yet for all his expertise, he was not able to educate people and fell prey to his own dogma which was unreconcilable with the unprecedented scaling challenges faced by the social web. No doubt he is still sitting in his office wildly gesticulating about the failure of the market to produce a “true” RDBMS system, which, of course, would have superceded the need for the current breed of alternate database technologies.
The lesson here, which I would have posted to Domas’ post if he allowed comments, is that no one knows everything, and if you are a domain expert you should share your knowledge openly rather than belittling others. If Domas built a web application I could poke his implementation choices full of countless holes, but it wouldn’t make me smarter than him.
I hail from DB2 land and I really enjoyed the ability to put one or more tables of my choosing in a particular tablespace and others in another tablespace.
I totally agree with many of the points here about the filesystem overhead of having a file per table. However, I also don’t want all tables to be in one file.
Is there not a happy medium in mysql?
I’ve been running several MySQL servers for a number of years. I have one particular box that has a little over 400 databases totaling about 350GB. The server was originally setup using 3 ibdata files back on 4.0, and was later changed to file per table. There has not been any appreciable speed difference from what I can see. The server is very busy, averaging 6-7 GB of binlogs per day. There is file system overhead for sure, but the db files are stored on a 16 spindle 6gb/s sas array, so I don’t expect that any small gain would warrant moving back to single files again.
@John, George,
Ah, but it has been more than two years since this post was published. Much has happened since.
In particular, the emergence of InnoDB Plugin (now just InnoDB), replacing the older “builting” InnoDB.
InnoDB Plugic comes with many improvements; performance, scale up, bugfixes, blob storage, …
And to fully utilize its functionality (you can use half of its advantages without changing anything), you need to upgrade your tables to “barracuda format”.
Guess what? It requires innodb_file_per_table.
So file-per-table is now also a feature issue. You loose features (e.g. compression, blob storage) which can directly relate to performance issues.