Package com.zendesk.maxwell.replication

Class BinlogConnectorReplicator

java.lang.Object

com.zendesk.maxwell.util.RunLoopProcess

com.zendesk.maxwell.replication.BinlogConnectorReplicator

All Implemented Interfaces:

com.github.shyiko.mysql.binlog.BinaryLogClient.LifecycleListener, Replicator, StoppableTask

public class BinlogConnectorReplicator
extends RunLoopProcess
implements Replicator, com.github.shyiko.mysql.binlog.BinaryLogClient.LifecycleListener

Field Summary

Fields

Modifier and Type	Field	Description
static int	ACCESS_DENIED_ERROR_CODE
static int	BAD_BINLOG_ERROR_CODE
static int	BINLOG_QUEUE_SIZE
protected com.github.shyiko.mysql.binlog.BinaryLogClient	client

Fields inherited from class com.zendesk.maxwell.util.RunLoopProcess

taskState

Constructor Summary

Constructors

Constructor	Description
BinlogConnectorReplicator(SchemaStore schemaStore, AbstractProducer producer, BootstrapController bootstrapper, MaxwellMysqlConfig mysqlConfig, Long replicaServerID, String maxwellSchemaDatabaseName, Metrics metrics, Position start, boolean stopOnEOF, String clientID, HeartbeatNotifier heartbeatNotifier, Scripting scripting, Filter filter, boolean ignoreMissingSchema, MaxwellOutputConfig outputConfig, float bufferMemoryUsage, int replicationReconnectionRetries, int binlogEventQueueSize)
BinlogConnectorReplicator(SchemaStore schemaStore, AbstractProducer producer, BootstrapController bootstrapper, MaxwellMysqlConfig mysqlConfig, Long replicaServerID, String maxwellSchemaDatabaseName, Metrics metrics, Position start, boolean stopOnEOF, String clientID, HeartbeatNotifier heartbeatNotifier, Scripting scripting, Filter filter, MaxwellOutputConfig outputConfig, float bufferMemoryUsage, int replicationReconnectionRetries)

Method Summary

Modifier and Type	Method	Description
protected void	beforeStop()
Long	getLastHeartbeatRead()	Get the last heartbeat that the replicator has processed.
RowMap	getRow()	The main entry point into the event reading loop.
Schema	getSchema()
Long	getSchemaId()
protected boolean	isMaxwellRow(RowMap row)	Is this RowMap an update to one of maxwell's own tables? We don't output updates to maxwell.positions, and updates to maxwell.heartbeats are always treated specially.
void	onCommunicationFailure(com.github.shyiko.mysql.binlog.BinaryLogClient client, Exception ex)	Listener for communication errors so we can stop everything and exit on this case
void	onConnect(com.github.shyiko.mysql.binlog.BinaryLogClient client)
void	onDisconnect(com.github.shyiko.mysql.binlog.BinaryLogClient client)
void	onEventDeserializationFailure(com.github.shyiko.mysql.binlog.BinaryLogClient client, Exception ex)
protected BinlogConnectorEvent	pollEvent()
protected void	processRow(RowMap row)
void	startReplicator()
void	stopAtHeartbeat(long heartbeat)
void	work()	get a single row from the replicator and pass it to the producer or bootstrapper.

Methods inherited from class com.zendesk.maxwell.util.RunLoopProcess

awaitStop, beforeStart, interrupt, requestStop, runLoop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.zendesk.maxwell.replication.Replicator

runLoop

Methods inherited from interface com.zendesk.maxwell.util.StoppableTask

awaitStop, requestStop

Field Detail

BINLOG_QUEUE_SIZE

public static int BINLOG_QUEUE_SIZE

BAD_BINLOG_ERROR_CODE

public static final int BAD_BINLOG_ERROR_CODE

See Also:

Constant Field Values

ACCESS_DENIED_ERROR_CODE

public static final int ACCESS_DENIED_ERROR_CODE

See Also:

Constant Field Values

client

protected final com.github.shyiko.mysql.binlog.BinaryLogClient client

Constructor Detail

BinlogConnectorReplicator

public BinlogConnectorReplicator(SchemaStore schemaStore,
                                 AbstractProducer producer,
                                 BootstrapController bootstrapper,
                                 MaxwellMysqlConfig mysqlConfig,
                                 Long replicaServerID,
                                 String maxwellSchemaDatabaseName,
                                 Metrics metrics,
                                 Position start,
                                 boolean stopOnEOF,
                                 String clientID,
                                 HeartbeatNotifier heartbeatNotifier,
                                 Scripting scripting,
                                 Filter filter,
                                 MaxwellOutputConfig outputConfig,
                                 float bufferMemoryUsage,
                                 int replicationReconnectionRetries)

BinlogConnectorReplicator

public BinlogConnectorReplicator(SchemaStore schemaStore,
                                 AbstractProducer producer,
                                 BootstrapController bootstrapper,
                                 MaxwellMysqlConfig mysqlConfig,
                                 Long replicaServerID,
                                 String maxwellSchemaDatabaseName,
                                 Metrics metrics,
                                 Position start,
                                 boolean stopOnEOF,
                                 String clientID,
                                 HeartbeatNotifier heartbeatNotifier,
                                 Scripting scripting,
                                 Filter filter,
                                 boolean ignoreMissingSchema,
                                 MaxwellOutputConfig outputConfig,
                                 float bufferMemoryUsage,
                                 int replicationReconnectionRetries,
                                 int binlogEventQueueSize)

Method Detail

work

public void work()
          throws Exception

get a single row from the replicator and pass it to the producer or bootstrapper. This is the top-level function in the run-loop.

Specified by:

work in class RunLoopProcess

Throws:

Exception

startReplicator

public void startReplicator()
                     throws Exception

Specified by:

startReplicator in interface Replicator

Throws:

Exception

beforeStop

protected void beforeStop()
                   throws Exception

Overrides:

beforeStop in class RunLoopProcess

Throws:

Exception

onCommunicationFailure

public void onCommunicationFailure(com.github.shyiko.mysql.binlog.BinaryLogClient client,
                                   Exception ex)

Listener for communication errors so we can stop everything and exit on this case

Specified by:

onCommunicationFailure in interface com.github.shyiko.mysql.binlog.BinaryLogClient.LifecycleListener

Parameters:

ex - Exception thrown by the BinaryLogClient

getLastHeartbeatRead

public Long getLastHeartbeatRead()

Get the last heartbeat that the replicator has processed. We pass along the value of the heartbeat to the producer inside the row map.

Specified by:

getLastHeartbeatRead in interface Replicator

Returns:

the millisecond value ot the last heartbeat read

stopAtHeartbeat

public void stopAtHeartbeat(long heartbeat)

Specified by:

stopAtHeartbeat in interface Replicator

processRow

protected void processRow(RowMap row)
                   throws Exception

Throws:

Exception

isMaxwellRow

protected boolean isMaxwellRow(RowMap row)

Is this RowMap an update to one of maxwell's own tables? We don't output updates to maxwell.positions, and updates to maxwell.heartbeats are always treated specially.

Parameters:

row - The RowMap in question

Returns:

whether the update is something maxwell itself generated

getRow

public RowMap getRow()
              throws Exception

The main entry point into the event reading loop.

We maintain a buffer of events in a transaction, and each subsequent call to `getRow` can grab one from the buffer. If that buffer is empty, we'll go check the open-replicator buffer for rows to process. If that buffer is empty, we return null.

Specified by:

getRow in interface Replicator

Returns:

either a RowMap or null

Throws:

Exception

pollEvent

protected BinlogConnectorEvent pollEvent()
                                  throws InterruptedException

Throws:

InterruptedException

getSchema

public Schema getSchema()
                 throws SchemaStoreException

Specified by:

getSchema in interface Replicator

Throws:

SchemaStoreException

getSchemaId

public Long getSchemaId()
                 throws SchemaStoreException

Specified by:

getSchemaId in interface Replicator

Throws:

SchemaStoreException

onConnect

public void onConnect(com.github.shyiko.mysql.binlog.BinaryLogClient client)

Specified by:

onConnect in interface com.github.shyiko.mysql.binlog.BinaryLogClient.LifecycleListener

onEventDeserializationFailure

public void onEventDeserializationFailure(com.github.shyiko.mysql.binlog.BinaryLogClient client,
                                          Exception ex)

Specified by:

onEventDeserializationFailure in interface com.github.shyiko.mysql.binlog.BinaryLogClient.LifecycleListener

onDisconnect

public void onDisconnect(com.github.shyiko.mysql.binlog.BinaryLogClient client)

Specified by:

onDisconnect in interface com.github.shyiko.mysql.binlog.BinaryLogClient.LifecycleListener