Hello, I was in need to use the replicator again, but this time I was able to create a minimum viable example.
Initial problem:
I was trying to replicate from a eventstore cloud instance on a public network to my local machine deploying the replicator on docker. But since the cloud instance is on 24.6 I can not connect to it via TCP. (which for me was the only way I was ever able to run the replicator).
I tested with grpc and was not able to make it work.
So in order to try to replicate on a simple example I created a docker compose to replicate locally so that I could share it here. I looked at the various documentations but I couldn’t find something to unblock me.
I am able to connect to both eventstore instances via Admin UI at localhost:2113 and localhost:4113, so the instances are reachable. This is the same docker setup I use to connect with my local application with grpc client with the same connection string using your java client.
Here is the docker compose:
version: '3.7'
services:
eventstore-source:
image: eventstore/eventstore:24.6.0-alpha-arm64v8
healthcheck:
test: [ "CMD", "curl", "-f", "http://localhost:2113/gossip" ]
interval: 10s
timeout: 10s
retries: 3
start_period: 5s
environment:
- EVENTSTORE_CLUSTER_SIZE=1
- EVENTSTORE_RUN_PROJECTIONS=All
- EVENTSTORE_START_STANDARD_PROJECTIONS=true
- EVENTSTORE_ENABLE_ATOM_PUB_OVER_HTTP=true
- EVENTSTORE_INSECURE=true
- EVENTSTORE_LOG_LEVEL=Verbose
volumes:
- './data-source:/var/lib/eventstore'
ports:
- 2113:2113
- 1113:1113
eventstore-target:
image: eventstore/eventstore:24.6.0-alpha-arm64v8
healthcheck:
test: [ "CMD", "curl", "-f", "http://localhost:2113/gossip" ]
interval: 10s
timeout: 10s
retries: 3
start_period: 5s
environment:
- EVENTSTORE_CLUSTER_SIZE=1
- EVENTSTORE_RUN_PROJECTIONS=All
- EVENTSTORE_START_STANDARD_PROJECTIONS=true
- EVENTSTORE_ENABLE_ATOM_PUB_OVER_HTTP=true
- EVENTSTORE_INSECURE=true
- EVENTSTORE_LOG_LEVEL=Verbose
volumes:
- './data-target:/var/lib/eventstore'
ports:
- 4113:2113
- 3113:1113
replicator:
container_name: repl-replicator
image: eventstore/replicator:0.4.8
depends_on:
eventstore-source:
condition: service_healthy
eventstore-target:
condition: service_healthy
ports:
- "5000:5000"
volumes:
- './replicator.yml:/app/config/appsettings.yaml'
environment:
REPLICATOR_DEBUG: 1
And my replicator.yml file:
replicator:
reader:
connectionString: "esdb://localhost:2113?tls=false"
protocol: grpc
sink:
connectionString: "esdb://localhost:4113?tls=false"
protocol: grpc
partitionCount: 1
scavenge: true
checkpoint:
path: "./checkpoint"
And this is the error message I get from the replicator:
"Starting gRPC reader","@i":"59fda7be","SourceContext":"EventStore.Replicator.Esdb.Grpc.GrpcEventReader"}
2025-02-06T16:34:37.964733718Z {"@t":"2025-02-06T16:34:37.9589695Z","@m":"Reader error","@i":"450d20cc","@l":"Error","@x":"Grpc.Core.RpcException: Status(StatusCode=\"Unavailable\", Detail=\"Error starting gRPC call. HttpRequestException: Connection refused (localhost:2113) SocketException: Connection refused\", DebugException=\"System.Net.Http.HttpRequestException: Connection refused (localhost:2113)\")\n ---> System.Net.Http.HttpRequestException: Connection refused (localhost:2113)\n ---> System.Net.Sockets.SocketException (111): Connection refused\n at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)\n at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|285_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken)\n at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)\n --- End of inner exception stack trace ---\n at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)\n at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)\n at System.Net.Http.HttpConnectionPool.AddHttp2ConnectionAsync(QueueItem queueItem)\n at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)\n at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)\n at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)\n at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)\n at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessage request, Nullable`1 timeout)\n --- End of inner exception stack trace ---\n at EventStore.Client.Interceptors.TypedExceptionInterceptor.<AsyncUnaryCall>b__5_0[TRequest,TResponse](Task`1 t)\n at System.Threading.Tasks.ContinuationResultTaskFromResultTask`2.InnerInvoke()\n at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)\n--- End of stack trace from previous location ---\n at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)\n at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)\n--- End of stack trace from previous location ---\n at EventStore.Client.GrpcServerCapabilitiesClient.GetAsync(CallInvoker callInvoker, CancellationToken cancellationToken)\n at EventStore.Client.EventStoreClientBase.GetChannelInfoExpensive(ReconnectionRequired reconnectionRequired, Action`1 onReconnectionRequired, IChannelSelector channelSelector, CancellationToken cancellationToken)\n at EventStore.Client.SharingProvider`2.FillBoxAsync(TaskCompletionSource`1 box, TInput input)\n at EventStore.Client.TaskExtensions.WithCancellation[T](Task`1 task, CancellationToken cancellationToken)\n at EventStore.Client.EventStoreClientBase.GetChannelInfo(CancellationToken cancellationToken)\n at EventStore.Client.EventStoreClient.<>c__DisplayClass25_0.<<ReadAllAsync>b__0>d.MoveNext()\n--- End of stack trace from previous location ---\n at EventStore.Client.EventStoreClient.ReadAllStreamResult.<>c__DisplayClass9_0.<<-ctor>g__PumpMessages|0>d.MoveNext()\n--- End of stack trace from previous location ---\n at System.Threading.Channels.AsyncOperation`1.GetResult(Int16 token)\n at System.Threading.Channels.ChannelReader`1.ReadAllAsync(CancellationToken cancellationToken)+MoveNext()\n at EventStore.Client.EventStoreClient.ReadAllStreamResult.GetAsyncEnumerator(CancellationToken cancellationToken)+MoveNext()\n at EventStore.Client.EventStoreClient.ReadAllStreamResult.GetAsyncEnumerator(CancellationToken cancellationToken)+MoveNext()\n at EventStore.Client.EventStoreClient.ReadAllStreamResult.GetAsyncEnumerator(CancellationToken cancellationToken)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()\n at Ubiquitous.Metrics.Metrics.MeasureValueTask[T](Func`1 action, IHistogramMetric metric, ICountMetric errorCount, String[] labels, Int32 count)\n at EventStore.Replicator.Esdb.Grpc.GrpcEventReader.ReadEvents(Position fromPosition, Func`2 next, CancellationToken cancellationToken) in /app/src/EventStore.Replicator.Esdb.Grpc/GrpcEventReader.cs:line 63\n at EventStore.Replicator.Read.ReaderPipe.<>c__DisplayClass1_0.<<-ctor>g__Reader|1>d.MoveNext() in /app/src/EventStore.Replicator/Read/ReaderPipe.cs:line 41","SourceContext":"EventStore.Replicator.Read.ReaderPipe"}
2025-02-06T16:34:37.964869718Z {"@t":"2025-02-06T16:34:37.9645543Z","@m":"Reader stopped","@i":"49a9146f","SourceContext":"EventStore.Replicator.Read.ReaderPipe"}
2025-02-06T16:34:37.964876343Z {"@t":"2025-02-06T16:34:37.9646955Z","@m":"Storing the last known checkpoint","@i":"228021e9","SourceContext":"EventStore.Replicator.Replicator"}
Thank you in advance for any help