We have a two sql publisher servers, configured for merge replication and each publication have 5 - 10 subscriptions, the replication was running fine, but suddenly it started giving the below error on two servers, but the error message is same for all the
failures on the two different servers,
Error messages:
The merge process was unable to change generation history at the 'Subscriber'. When troubleshooting, restart the synchronization with verbose history logging and specify an output file to which to write. (Source: MSSQL_REPL, Error number: MSSQL_REPL-2147200995)
Get help: http://help/MSSQL_REPL-2147200995
TCP Provider: The semaphore timeout period has expired. (Source: MSSQLServer, Error number: 121)
Get help: http://help/121
Communication link failure (Source: MSSQLServer, Error number: 121)
Get help: http://help/121
I enabled merge agent for logging to output file with -OutputVerboseLevel 4 and also with -OutputVerboseLevel 3, both generated a similar log and gave me the same error message as above what we collected from the replication monitor.
2014-10-16 13:45:10.975 Microsoft SQL Server Merge Agent 11.0.5058.0
2014-10-16 13:45:11.006 Copyright (c) 2008 Microsoft Corporation
2014-10-16 13:45:11.019
2014-10-16 13:45:11.026 The timestamps prepended to the output lines are expressed in terms of UTC time.
2014-10-16 13:45:11.031 User-specified agent parameter values:
-Publisher SVRGBBRYSQL01
-PublisherDB CommonDatabase
-Publication CommonDatabase_IS_AS
-Subscriber ASNDAMRKT01
-SubscriberDB CommonDatabase
-Distributor SVRGBBRYSQL01
-DistributorSecurityMode 1
-Output C:\ReplOutput1.txt
-OutputVerboseLevel 4
-QueryTimeOut 3600
-DownloadGenerationsPerBatch 2000
-UploadGenerationsPerBatch 2000
2014-10-16 13:45:11.091 Percent Complete: 0
2014-10-16 13:45:11.095 Connecting to Distributor 'SVRGBBRYSQL01'
2014-10-16 13:45:11.099 Repl Agent Status: 3
2014-10-16 13:45:11.104 Connecting to OLE DB Distributor at datasource: 'SVRGBBRYSQL01', location: '', catalog: '', providerstring: '' using provider 'SQLNCLI11'
2014-10-16 13:45:11.160 OLE DB Distributor: SVRGBBRYSQL01
DBMS: Microsoft SQL Server
Version: 11.00.5058
catalog name:
user name: dbo
API conformance: 0
SQL conformance: 0
transaction capable: 1
read only: F
identifier quote char: "
non_nullable_columns: 0
owner usage: 15
max table name len: 128
max column name len: 128
need long data len:
max columns in table: 1000
max columns in index: 16
max char literal len: 131072
max statement len: 131072
max row size: 131072
2014-10-16 13:45:11.177 OLE DB Distributor 'SVRGBBRYSQL01': select SERVERPROPERTY ('ProductVersion')
2014-10-16 13:45:11.186 OLE DB Distributor 'SVRGBBRYSQL01': {call sp_helpdistpublisher (N'SVRGBBRYSQL01') }
2014-10-16 13:45:11.209 OLE DB Distributor 'SVRGBBRYSQL01': {call sp_MShelp_repl_agent (N'SVRGBBRYSQL01', N'CommonDatabase', N'CommonDatabase_IS_AS', N'ASNDAMRKT01', N'CommonDatabase', 1)}
2014-10-16 13:45:11.218 OLE DB Distributor 'SVRGBBRYSQL01': select datasource, srvid from master..sysservers where upper(srvname) = upper(N'SVRGBBRYSQL01')
2014-10-16 13:45:11.234 OLE DB Distributor 'SVRGBBRYSQL01': {call sp_MShelp_merge_agentid (0,N'CommonDatabase',N'CommonDatabase_IS_AS',null,N'CommonDatabase',90,N'ASNDAMRKT01')}
2014-10-16 13:45:11.241 OLE DB Distributor 'SVRGBBRYSQL01': {call sp_MShelp_profile (74, 4, N'')}
2014-10-16 13:45:11.247 OLE DB Distributor 'SVRGBBRYSQL01': {call sys.sp_get_redirected_publisher(N'SVRGBBRYSQL01',N'CommonDatabase',0)}
2014-10-16 13:45:11.253 Percent Complete: 0
2014-10-16 13:45:11.254 Connecting to OLE DB Publisher at datasource: 'SVRGBBRYSQL01', location: '', catalog: 'CommonDatabase', providerstring: '' using provider 'SQLNCLI11'
2014-10-16 13:45:11.257 Initializing
2014-10-16 13:45:11.266 Repl Agent Status: 1
2014-10-16 13:45:11.272 OLE DB Distributor 'SVRGBBRYSQL01': {call sys.sp_MSadd_merge_history90 (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)}
2014-10-16 13:45:11.273 OLE DB Publisher: SVRGBBRYSQL01
DBMS: Microsoft SQL Server
Version: 11.00.5058
catalog name: CommonDatabase
user name: dbo
API conformance: 0
SQL conformance: 0
transaction capable: 1
read only: F
identifier quote char: "
non_nullable_columns: 0
owner usage: 15
max table name len: 128
max column name len: 128
need long data len:
max columns in table: 1000
max columns in index: 16
max char literal len: 131072
max statement len: 131072
max row size: 131072
2014-10-16 13:45:11.282 Percent Complete: 0
2014-10-16 13:45:11.283 OLE DB Publisher 'SVRGBBRYSQL01': set nocount on declare @dbname sysname select @dbname = db_name() declare @collation nvarchar(255) select @collation = convert(nvarchar(255), databasepropertyex(@dbname, N'COLLATION')) select collationproperty(@collation,
N'CODEPAGE') as 'CodePage', collationproperty(@collation, N'LCID') as 'LCID', collationproperty(@collation, N'COMPARISONSTYLE') as 'ComparisonStyle',cast(case when convert (int,databasepropertyex (@dbname,'comparisonstyle')) & 0x1 = 0x1 then 0 else 1 end
as bit) as DB_CaseSensitive,cast(case when convert (int,serverproperty ('comparisonstyle')) & 0x1 = 0x1 then 0 else 1 end as bit) as Server_CaseSensitive set nocount off
2014-10-16 13:45:11.288 Validating publisher
2014-10-16 13:45:11.299 OLE DB Publisher 'SVRGBBRYSQL01': select SERVERPROPERTY ('ProductVersion')
2014-10-16 13:45:11.303 Repl Agent Status: 1
2014-10-16 13:45:11.310 Connecting to OLE DB Publisher at datasource: 'SVRGBBRYSQL01', location: '', catalog: 'CommonDatabase', providerstring: '' using provider 'SQLNCLI11'
2014-10-16 13:45:11.314 OLE DB Distributor 'SVRGBBRYSQL01': {call sys.sp_MSadd_merge_history90 (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)}
2014-10-16 13:45:11.329 OLE DB Publisher: SVRGBBRYSQL01
DBMS: Microsoft SQL Server
Version: 11.00.5058
catalog name: CommonDatabase
user name: dbo
API conformance: 0
SQL conformance: 0
transaction capable: 1
read only: F
identifier quote char: "
non_nullable_columns: 0
owner usage: 15
max table name len: 128
max column name len: 128
need long data len:
max columns in table: 1000
max columns in index: 16
max char literal len: 131072
max statement len: 131072
max row size: 131072
2014-10-16 13:45:11.332 Percent Complete: 0
2014-10-16 13:45:11.340 Connecting to Publisher 'SVRGBBRYSQL01'
2014-10-16 13:45:11.343 Repl Agent Status: 3
2014-10-16 13:45:11.347 OLE DB Distributor 'SVRGBBRYSQL01': {call sys.sp_MSadd_merge_history90 (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)}
2014-10-16 13:45:11.351 OLE DB Publisher 'SVRGBBRYSQL01': {call sp_MSchecksnapshotstatus (N'CommonDatabase_IS_AS')}
2014-10-16 13:45:11.358 OLE DB Publisher 'SVRGBBRYSQL01': {call sp_helpmergepublication (N'CommonDatabase_IS_AS')}
2014-10-16 13:45:11.370 OLE DB Publisher 'SVRGBBRYSQL01': {call sys.sp_MSgetreplicainfo(?,?,?,?,?,?,?,90)}
2014-10-16 13:45:11.377 OLE DB Distributor 'SVRGBBRYSQL01': {call sp_MShelp_repl_agent (N'SVRGBBRYSQL01', N'CommonDatabase', N'CommonDatabase_IS_AS', N'ASNDAMRKT01', N'CommonDatabase', 1)}
2014-10-16 13:45:11.385 Connecting to OLE DB Subscriber at datasource: 'ASNDAMRKT01', location: '', catalog: 'CommonDatabase', providerstring: '' using provider 'SQLNCLI11'
2014-10-16 13:45:11.627 OLE DB Subscriber: ASNDAMRKT01
DBMS: Microsoft SQL Server
Version: 10.00.5520
catalog name: CommonDatabase
user name: dbo
API conformance: 0
SQL conformance: 0
transaction capable: 1
read only: F
identifier quote char: "
non_nullable_columns: 0
owner usage: 15
max table name len: 128
max column name len: 128
need long data len:
max columns in table: 1000
max columns in index: 16
max char literal len: 131072
max statement len: 131072
max row size: 131072
2014-10-16 13:45:11.817 OLE DB Subscriber 'ASNDAMRKT01': select SERVERPROPERTY ('ProductVersion')
2014-10-16 13:45:11.854 OLE DB Subscriber 'ASNDAMRKT01': {call sp_MSreplcheck_subscribe}
2014-10-16 13:45:11.916 OLE DB Subscriber 'ASNDAMRKT01': set nocount on declare @dbname sysname select @dbname = db_name() declare @collation nvarchar(255) select @collation = convert(nvarchar(255), databasepropertyex(@dbname, N'COLLATION')) select collationproperty(@collation,
N'CODEPAGE') as 'CodePage', collationproperty(@collation, N'LCID') as 'LCID', collationproperty(@collation, N'COMPARISONSTYLE') as 'ComparisonStyle',cast(case when convert (int,databasepropertyex (@dbname,'comparisonstyle')) & 0x1 = 0x1 then 0 else 1 end
as bit) as DB_CaseSensitive,cast(case when convert (int,serverproperty ('comparisonstyle')) & 0x1 = 0x1 then 0 else 1 end as bit) as Server_CaseSensitive set nocount off
2014-10-16 13:45:11.993 Percent Complete: 0
2014-10-16 13:45:11.993 OLE DB Subscriber 'ASNDAMRKT01': {call sys.sp_MSadd_merge_history90 (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)}
2014-10-16 13:45:12.002 Connecting to Subscriber 'ASNDAMRKT01'
2014-10-16 13:45:12.022 Repl Agent Status: 3
2014-10-16 13:45:12.026 OLE DB Distributor 'SVRGBBRYSQL01': {call sys.sp_MSadd_merge_history90 (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)}
2014-10-16 13:45:12.049 Percent Complete: 0
2014-10-16 13:45:12.050 OLE DB Subscriber 'ASNDAMRKT01': {call sys.sp_MSadd_merge_history90 (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)}
2014-10-16 13:45:12.054 Retrieving publication information
2014-10-16 13:45:12.063 Repl Agent Status: 3
2014-10-16 13:45:12.067 OLE DB Distributor 'SVRGBBRYSQL01': {call sys.sp_MSadd_merge_history90 (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)}
2014-10-16 13:45:12.104 OLE DB Subscriber 'ASNDAMRKT01': {call sys.sp_MSmerge_upgrade_subscriber(1,?)}
2014-10-16 13:45:12.105 Percent Complete: 0
2014-10-16 13:45:12.113 Retrieving subscription information.
2014-10-16 13:45:12.117 Repl Agent Status: 3
2014-10-16 13:45:12.121 OLE DB Distributor 'SVRGBBRYSQL01': {call sys.sp_MSadd_merge_history90 (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)}
2014-10-16 13:45:12.445 OLE DB Publisher 'SVRGBBRYSQL01': {call sys.sp_MSgetreplicainfo(?,?,?,?,?,?,?,90)}
2014-10-16 13:45:12.457 OLE DB Subscriber 'ASNDAMRKT01': {call sys.sp_MSgetreplicainfo(?,?,?,?,?,?,?,90)}
2014-10-16 13:45:12.507 OLE DB Publisher 'SVRGBBRYSQL01': {call sys.sp_MShelpmergearticles (?,10000000,?) }
2014-10-16 13:45:13.023 OLE DB Publisher 'SVRGBBRYSQL01': {call sp_MSenumschemachange (?,?,10000000,?,0,1) }
2014-10-16 13:45:13.044 OLE DB Subscriber 'ASNDAMRKT01': {call sys.sp_MSgetreplicainfo(?,?,?,?,?,?,?,90)}
2014-10-16 13:45:13.107 OLE DB Publisher 'SVRGBBRYSQL01': {call sys.sp_MSsetreplicainfo (?,?,?,?,?,?,?,?,?,?,90,?,?,1)}
2014-10-16 13:45:13.130 OLE DB Subscriber 'ASNDAMRKT01': {call sp_MSmergeupdatelastsyncinfo (?,?,?)}
2014-10-16 13:45:13.184 OLE DB Subscriber 'ASNDAMRKT01': {call sys.sp_MSenumreplicas90 }
2014-10-16 13:45:13.294 OLE DB Publisher 'SVRGBBRYSQL01': {call sys.sp_MSenumreplicas90 }
2014-10-16 13:45:13.322 OLE DB Subscriber 'ASNDAMRKT01': {call sp_MScreateglobalreplica (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,90)}
2014-10-16 13:45:13.364 OLE DB Subscriber 'ASNDAMRKT01': {call sp_MScreateglobalreplica (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,90)}
2014-10-16 13:45:13.410 OLE DB Publisher 'SVRGBBRYSQL01': {call sp_MScreateglobalreplica (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,90)}
2014-10-16 13:45:13.421 OLE DB Publisher 'SVRGBBRYSQL01': {call sp_MScreateglobalreplica (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,90)}
2014-10-16 13:45:13.433 OLE DB Publisher 'SVRGBBRYSQL01': {call sp_MScreateglobalreplica (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,90)}
2014-10-16 13:45:13.445 OLE DB Publisher 'SVRGBBRYSQL01': {call sp_MScreateglobalreplica (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,90)}
2014-10-16 13:45:13.455 OLE DB Subscriber 'ASNDAMRKT01': {call sp_helpmergealternatepublisher(N'SVRGBBRYSQL01', N'CommonDatabase', N'CommonDatabase_IS_AS')}
2014-10-16 13:45:13.495 OLE DB Publisher 'SVRGBBRYSQL01': {call sp_helpmergealternatepublisher(N'SVRGBBRYSQL01', N'CommonDatabase', N'CommonDatabase_IS_AS')}
2014-10-16 13:45:13.511 OLE DB Subscriber 'ASNDAMRKT01': {call sys.sp_MSenum_metadataaction_requests(?,?,?,?)}
2014-10-16 13:45:13.561 OLE DB Subscriber 'ASNDAMRKT01': {call sp_MSgetlastsentrecgens (?)}
2014-10-16 13:45:13.599 OLE DB Publisher 'SVRGBBRYSQL01': {call sp_MSgetlastsentrecgens (?)}
2014-10-16 13:45:13.605 OLE DB Subscriber 'ASNDAMRKT01': {call sys.sp_MSadd_merge_history90 (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)}
2014-10-16 13:45:13.662 Percent Complete: 0
2014-10-16 13:45:13.662 OLE DB Subscriber 'ASNDAMRKT01': {call sp_MSgetmakegenerationapplock_90(?,?)}
2014-10-16 13:45:13.668 Uploading data changes to the Publisher
2014-10-16 13:45:13.679 Repl Agent Status: 3
2014-10-16 13:45:13.688 OLE DB Distributor 'SVRGBBRYSQL01': {call sys.sp_MSadd_merge_history90 (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)}
2014-10-16 13:45:13.711 OLE DB Subscriber 'ASNDAMRKT01': {call sys.sp_MSmakegeneration (?, NULL,NULL,NULL,100)}
2014-10-16 13:45:14.209 OLE DB Subscriber 'ASNDAMRKT01': {call sp_MSreleasemakegenerationapplock}
2014-10-16 13:45:14.269 OLE DB Subscriber 'ASNDAMRKT01': {call sys.sp_MSenumgenerations90 (5201223,?,0,?,?,?) }
2014-10-16 13:45:14.318 OLE DB Publisher 'SVRGBBRYSQL01': {call sp_MScheckexistsgeneration (?, ?) }
2014-10-16 13:45:14.325 OLE DB Subscriber 'ASNDAMRKT01': {call sp_MSsetlastsentgen (?, ?, ?)}
2014-10-16 13:45:14.384 OLE DB Publisher 'SVRGBBRYSQL01': {call sp_MSsetlastrecgen (?, ?, ?)}
2014-10-16 13:45:14.403 OLE DB Subscriber 'ASNDAMRKT01': {call sys.sp_MSadd_merge_history90 (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)}
2014-10-16 13:45:14.475 Percent Complete: 0
2014-10-16 13:45:14.475 OLE DB Subscriber 'ASNDAMRKT01': {call sys.sp_MSadd_merge_history90 (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)}
2014-10-16 13:45:14.484 No data needed to be merged.
2014-10-16 13:45:14.497 Repl Agent Status: 3
2014-10-16 13:45:14.503 OLE DB Distributor 'SVRGBBRYSQL01': {call sys.sp_MSadd_merge_history90 (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)}
2014-10-16 13:45:14.530 OLE DB Publisher 'SVRGBBRYSQL01': {call sys.sp_MSgetreplicastate (?,?,?)}
2014-10-16 13:45:14.538 Percent Complete: 0
2014-10-16 13:45:14.547 OLE DB Subscriber 'ASNDAMRKT01': {call sys.sp_MSgetreplicastate (?,?,?)}
2014-10-16 13:45:14.554 Downloading data changes to the Subscriber
2014-10-16 13:45:14.580 Repl Agent Status: 3
2014-10-16 13:45:14.591 OLE DB Distributor 'SVRGBBRYSQL01': {call sys.sp_MSadd_merge_history90 (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)}
2014-10-16 13:45:14.595 OLE DB Publisher 'SVRGBBRYSQL01': {call sys.sp_MSgetalternaterecgens (?)}
2014-10-16 13:45:14.612 OLE DB Publisher 'SVRGBBRYSQL01': {call sys.sp_is_makegeneration_needed(?, ?)}
2014-10-16 13:45:14.627 OLE DB Publisher 'SVRGBBRYSQL01': {call sp_MSgetmakegenerationapplock_90(?,?)}
2014-10-16 13:45:19.637 OLE DB Publisher 'SVRGBBRYSQL01': {call sp_MSgetmakegenerationapplock_90(?,?)}
2014-10-16 13:45:24.644 OLE DB Publisher 'SVRGBBRYSQL01': {call sp_MSgetmakegenerationapplock_90(?,?)}
2014-10-16 13:45:29.650 OLE DB Publisher 'SVRGBBRYSQL01': {call sp_MSgetmakegenerationapplock_90(?,?)}
2014-10-16 13:45:34.666 OLE DB Publisher 'SVRGBBRYSQL01': {call sp_MSgetmakegenerationapplock_90(?,?)}
2014-10-16 13:45:39.678 OLE DB Publisher 'SVRGBBRYSQL01': {call sp_MSgetmakegenerationapplock_90(?,?)}
2014-10-16 13:45:44.692 OLE DB Publisher 'SVRGBBRYSQL01': {call sp_MSgetmakegenerationapplock_90(?,?)}
2014-10-16 13:45:44.697 OLE DB Publisher 'SVRGBBRYSQL01': {call sp_MSreleasemakegenerationapplock}
2014-10-16 13:45:44.703 OLE DB Publisher 'SVRGBBRYSQL01': {call sys.sp_MSenumgenerations90 (8489377,?,0,?,?,?) }
2014-10-16 13:45:44.740 OLE DB Subscriber 'ASNDAMRKT01': {call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?,
?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call
sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration
(?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call
sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration
(?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call
sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration
(?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call
sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration
(?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call
sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration
(?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call
sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration
(?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call
sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration
(?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call
sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration
(?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call
sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration
(?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call
sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration
(?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call
sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration
(?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call
sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration
(?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call
sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration
(?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call
sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration
(?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call sp_MScheckexistsgeneration (?, ?) }{call
sp_MScheckexistsgeneration (?, ?) }{call sp_MScThe merge process was unable to verify the existence of a generation at the 'Subscriber'. If this failure persists, reinitialize the subscription.
2014-10-16 13:46:03.866 OLE DB Subscriber 'ASNDAMRKT01': {call sys.sp_MSadd_merge_history90 (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)}
2014-10-16 13:46:07.192 Percent Complete: 0
2014-10-16 13:46:07.192 OLE DB Subscriber 'ASNDAMRKT01': {call sys.sp_MSadd_merge_history90 (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)}
2014-10-16 13:46:07.197 No data needed to be merged.
2014-10-16 13:46:07.208 Repl Agent Status: 3
2014-10-16 13:46:07.213 OLE DB Distributor 'SVRGBBRYSQL01': {call sys.sp_MSadd_merge_history90 (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)}
2014-10-16 13:46:07.440 Percent Complete: 0
2014-10-16 13:46:07.444 The merge process was unable to verify the existence of a generation at the 'Subscriber'. If this failure persists, reinitialize the subscription.
2014-10-16 13:46:07.449 Repl Agent Status: 6
2014-10-16 13:46:07.454 OLE DB Distributor 'SVRGBBRYSQL01': {call sys.sp_MSadd_merge_history90 (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)}
2014-10-16 13:46:07.500 OLE DB Subscriber 'ASNDAMRKT01': {call sp_MSmergeupdatelastsyncinfo (?,?,?)}
2014-10-16 13:46:07.506 The Merge Agent was unable to update information about the last synchronization at the Subscriber. Ensure that the subscription exists at the Subscriber, and restart the Merge Agent.
2014-10-16 13:46:07.768 Percent Complete: 0
2014-10-16 13:46:07.774 Category:NULL
Source: Merge Replication Provider
Number: -2147201003
Message: The merge process was unable to verify the existence of a generation at the 'Subscriber'. If this failure persists, reinitialize the subscription.
2014-10-16 13:46:07.779 Repl Agent Status: 3
2014-10-16 13:46:08.025 Percent Complete: 0
2014-10-16 13:46:08.030 Category:SQLSERVER
Source: ASNDAMRKT01
Number: 121
Message: TCP Provider: The semaphore timeout period has expired.
2014-10-16 13:46:08.036 Repl Agent Status: 3
2014-10-16 13:46:08.956 Percent Complete: 0
2014-10-16 13:46:08.961 Category:SQLSERVER
Source: ASNDAMRKT01
Number: 121
Message: Communication link failure
2014-10-16 13:46:08.965 Repl Agent Status: 3
**************************************************************
CONNECTION TIMES
Publisher (all connections)
109 msec
Subscriber (all connections)
562 msec
Distributor
63 msec
**************************************************************
UPLOAD COUNTERS
MakeGeneration Time = 608 msec.
InsertGenHistory Time = 0 msec.
UpdateGenHistory Time = 0 msec.
ProxiedMetadata Time = 0 msec.
**************************************************************
DOWNLOAD COUNTERS
MakeGeneration Time = 30077 msec.
InsertGenHistory Time = 19126 msec.
UpdateGenHistory Time = 0 msec.
**************************************************************
RETENTION-BASED CLEANUP STATISTICS
Publisher:
Cleanup Time
0 msec
MSmerge_genhistory rows cleaned up
0
MSmerge_contents rows cleaned up
0
MSmerge_tombstone rows cleaned up
0
Subscriber:
Cleanup Time
0 msec
MSmerge_genhistory rows cleaned up
0
MSmerge_contents rows cleaned up
0
MSmerge_rowtrack rows cleaned up
0
MSmerge_tombstone rows cleaned up
0
**************************************************************
RETRY STATISTICS
Retry Time (Upload)
0 msec.
Retry Time (Download)
0 msec.
Total changes retried
0
Number of Iterations through rows needing retry
0
Total number of changes that failed despite retry
0
**************************************************************
PROXY METADATA QUEUE COUNTERS
Queue Full:
Number of Waits: 0, Total Wait Time: 0 msec
**************************************************************
Distributor-side History Logging Time = 514 msec.
Number of Distributor-side History Messages Logged = 11
Subscriber-side History Logging Time = 3870 msec.
Number of Subscriber-side History Messages Logged = 11
**************************************************************
2014-10-16 13:46:08.985 OLE DB Publisher 'SVRGBBRYSQL01': DBCC SQLPERF (NETSTATS)
2014-10-16 13:46:08.991 OLE DB Subscriber 'ASNDAMRKT01': DBCC SQLPERF (NETSTATS)
2014-10-16 13:46:09.811 Connecting to OLE DB Subscriber at datasource: 'ASNDAMRKT01', location: '', catalog: 'CommonDatabase', providerstring: '' using provider 'SQLNCLI11'
2014-10-16 13:46:10.375 OLE DB Subscriber 'ASNDAMRKT01': DBCC SQLPERF (NETSTATS)
2014-10-16 13:46:10.604 OLE DB Distributor 'SVRGBBRYSQL01': DBCC SQLPERF (NETSTATS)
NETWORK STATISTICS
Server
Reads
Writes
Bytes Read
Bytes Written
Publisher
4369
4881
1291377
8118830
Subscriber
521
5174
253591
19505410
Distributor
4540
5056
1340102
8301202
**************************************************************
NETWORK STATUS
Network Connection:
The computer has one or more LAN cards that are active.
Network link speed:
Destination
Incoming
Outgoing
Publisher
Unreachable
Unreachable
Subscriber
Unreachable
Unreachable
Distributor
Unreachable
Unreachable
**************************************************************
Duration of synchronization session:
59 sec
2014-10-16 13:46:10.633 OLE DB Subscriber 'ASNDAMRKT01': {call sp_MSmergeupdatelastsyncinfo (?,?,?)}
2014-10-16 13:46:10.633 [100%] Percent Complete: 100
2014-10-16 13:46:10.647 Thread id 7948 is attempting to re-establish the connection to Subscriber 'ASNDAMRKT01'.
2014-10-16 13:46:10.653 Repl Agent Status: 3
2014-10-16 13:46:10.661 OLE DB Distributor 'SVRGBBRYSQL01': {call sys.sp_MSadd_merge_history90 (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)}
2014-10-16 13:46:10.671 [100%] Percent Complete: 100
2014-10-16 13:46:10.678 Thread id 7948 successfully re-established connection to Subscriber 'ASNDAMRKT01'.
2014-10-16 13:46:10.684 Repl Agent Status: 3
2014-10-16 13:46:10.688 OLE DB Distributor 'SVRGBBRYSQL01': {call sys.sp_MSadd_merge_history90 (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)}
2014-10-16 13:46:10.704 OLE DB Publisher 'SVRGBBRYSQL01': {call sp_MSmergeupdatelastsyncinfo (?,?,?)}
2014-10-16 13:46:10.713 Disconnecting from OLE DB Publisher 'SVRGBBRYSQL01'
2014-10-16 13:46:10.719 Disconnecting from OLE DB Publisher 'SVRGBBRYSQL01'
2014-10-16 13:46:10.724 Disconnecting from OLE DB Subscriber 'ASNDAMRKT01'
2014-10-16 13:46:10.730 Disconnecting from OLE DB Subscriber 'ASNDAMRKT01'
2014-10-16 13:46:10.735 Disconnecting from OLE DB Publisher 'SVRGBBRYSQL01'
2014-10-16 13:46:10.744 Disconnecting from OLE DB Publisher 'SVRGBBRYSQL01'
2014-10-16 13:46:10.750 Disconnecting from OLE DB Distributor 'SVRGBBRYSQL01'
2014-10-16 13:46:10.771 Disconnecting from OLE DB Distributor 'SVRGBBRYSQL01'
2014-10-16 13:46:10.776 The merge process will restart after waiting 30 second(s)...
Percent Complete: 0
I looked for lines with genstatus 3 and 4 in the table Msmerge_genhistory on both publisher and subscriber and didn't get any records. Also tried tweaking -DownloadGenerationsPerBatch and -UploadGenerationsPerBatch with higher and lower values
but of no use.
Suresh Babu