Comments on: Notes on row based replication

By: shlomi

shlomi — Tue, 22 Oct 2013 19:53:59 +0000

@Rhys,
Very nice; good to know this variable exists.

By: Rhys

Rhys — Mon, 21 Oct 2013 16:16:21 +0000

RE: The limitation / feature mentioned in Forgiveness. There is the slave_exec_mode variable to get around this.

http://dev.mysql.com/doc/refman/5.1/en/replication-rbr-usage.html

Probably not a good idea though.

By: shlomi

shlomi — Tue, 17 Jul 2012 05:59:19 +0000

@Rick,
afaik there is an internal row_id for InnoDB. I may be wrong on this.

By: Rick James

Rick James — Mon, 16 Jul 2012 22:47:35 +0000

@James,
Replication is blocked during a large SBR DELETE. This is an important reason for chunking big deletes.

@James, etc,
There is no “row_id” for InnoDB tables. It depends on the PRIMARY KEY. If you do not provide such, then one is provided for you. (And I guess it could ‘drift’.)

Deleting in batches:
http://mysql.rjweb.org/doc.php/deletebig
Usually an ID range is sufficient to be sure the Slave(s) are deleting the same batch. (There is no way to suppress the warning, even when it is clearly wrong.)

The best way to do a big delete is via DROP PARTITION, if your table can be structured to take advantage of such. It is instantaneous, atomic, etc.

@XL,
The shadow table is very efficient, but it can have a flaw — what if row(s) are INSERTed/DELETEd/UPDATEd during the ‘copy’? One solution, albeit clumsy, is a TRIGGER.

@Shlomi,
Sorry, I guess we hijacked your RBR discussion to talk about DELETEs.

By: XL

XL — Mon, 16 Jul 2012 17:24:58 +0000

If one has to delete all rows in a table, then one should use TRUNCATE TABLE. RBR treats this statement as DDL and thus logs it verbatim.

For session tables one often has this situation: delete all but some 1000 of a million rows. This should be implemented as:

create shadow table like the original
copy rows to keep
exchange tables by RENAME TABLE

Again this case is handled efficiently on the slave.

By: Jervin R

Jervin R — Sun, 15 Jul 2012 02:43:44 +0000

Of course one also has to note that if you do ‘DELETE FROM t’ on a table with 1m rows it will be 1m events on the slave too – I’ve seen slaves not able to keep up with that and so SESSION based statement format comes in handy or try MIXED 🙂

By: Cédric

Cédric — Mon, 09 Jul 2012 10:52:22 +0000

With RBR, you also need to have stricly the same definition for your columns (data type for exemple)

By: James Day

James Day — Wed, 04 Jul 2012 19:56:02 +0000

Shlomi, I don’t know of anything that would guarantee that the InnoDB row ID on a slave is the same as that of the same logical row on a master. If the slave is created by cloning binary files it will start that way but different ordering of inserts and deletes on master and slave is something that I expect to cause them to rapidly become out of sync. If the slave is created from a mysqldump backup they could start out very different.

There have been suggestions to consider using the InnoDB row ID but those practical difficulties have so far blocked it. For some installations it might be a useful clue that could be rejected if the rest of the row turns out not to match.

Deleting in batches is a good approach. Depending on the application multiple batches can be used. Say very different WHERE clauses so no part of the range can overlap and let one delete block another.

Smaller batches also have the advantage of making it easier for the purge thread to keep up and helping to keep all of the undo log records in the buffer pool, much faster than accumulating a large batch that can then become disk-bound if they have been flushed from the pool.

No concurrency benefit for a slave from multiple batches on the slave at the moment, maybe in the future.

By: shlomi

shlomi — Wed, 04 Jul 2012 17:23:30 +0000

@Daniël,
Thanks

@Robert,
Thanks – this is an interesting issue since I almost don’t do large DELETEs anymore – when they’re large, I split them into smaller chunks, like 1,000 at a time, and do many steps. This makes for smaller transactions, albeit separate ones.
It is this very scenario where I see an improvement with RBR.

This is not to say there are no cases where large DELETEs are in place. I’m just not too familiar with these.

At any case I did my best to sound vague and ambiguous in the last paragraph so that no one could blame me for being incorrect 😉

By: Robert Hodges

Robert Hodges — Wed, 04 Jul 2012 16:22:25 +0000

Hi Shlomi! On your third point it’s not clear that there is always less slave effort. For instance, running a large DELETE with a WHERE can be more efficient than repeating the DELETE for each row. This effect is very pronounced on large tables.