In a polyglot microservice stack-Go, Java, Node, and Python talking over Kafka and gRPC-you quickly hit the same problem: who owns the .proto files, the topic lists, and the XML or JSON that multiple services must agree on? Copying files into every repo guarantees drift. Git submodules help a little but still feel heavy for small, frequent contract changes.
In one real engagement the landscape was several dozen microservices spread across multiple languages and runtimes. There was no practical way to keep message types, Kafka topics, named constants, and related contract files consistent if each team maintained its own copy. What we needed was one system of record: a place where those definitions are stored once, then fetched by each service and turned into code automatically-whether that is generated types from Protobuf, topic constants from JSON, or other outputs per language. That model makes it straightforward to keep every consumer aligned with the same source of truth, instead of reconciling dozens of repos by hand.
One pragmatic pattern is to treat schema and artifact storage like any other shared dependency: publish versioned blobs to a schema registry and let each service fetch and code-generate in CI and locally. Apicurio Registry fits this role well: it exposes a stable HTTP API, supports many artifact types (Protobuf, JSON, OpenAPI, and more), and versions each artifact independently.
This post walks through why teams adopt this, what you gain, and how it can look in practice: a central definitions repository that publishes into Apicurio, and a consumer service (for example in Go) that pulls those definitions, runs Buf, and generates code before build-patterns that have been used successfully in production.
Single source of truth
Contracts live in one place (your definitions repo and/or the registry versions), not in N copies across N services-essential when many microservices and many languages are in play and everyone must agree on the same types, topics, and constants.
Polyglot by design
The registry stores bytes and metadata; consumers pick their own codegen (Buf for Go/Java, protoc elsewhere, hand-rolled generators for JSON). The registry does not force one language.
Explicit versioning
Each publish creates a new artifact version. Consumers can pin, resolve “latest for my branch”, or fall back to a known baseline-depending on how you implement resolution.
CI-friendly
Fetch → generate → compile is a straight line: curl (or a thin Make wrapper) plus your chosen generators, then your normal test and build steps.
Separation of concerns
Platform or domain teams maintain what is published; application teams maintain how they compile and use it-without forking proto trees.
Apicurio’s REST API (/apis/registry/v2/...) is what makes scripted publish and fetch straightforward from Make, GitHub Actions, or any other runner.
| Capability | How it helps |
|---|---|
| Groups and artifacts | Namespacing (groupId / artifactId) matches how you think about bounded contexts (shared envelopes vs. domain-specific Kafka events). |
| Typed artifacts | You can mark content as PROTOBUF, JSON, etc., so the right Content-Type and metadata go with each upload. |
| Version history | Every change is a new version; you can diff or roll forward intentionally. |
| Auth via API key | CI uses a secret; humans can use the same pattern locally. |
A dedicated repository holds canonical files under definitions/…-Protobuf sources, topics.json for Kafka names, and optionally WSDL, XSD, Schematron, and more. A manifest (artifacts.json) maps each file to a group, artifact id, and artifact type so automation knows what to upload and how.
Typical CI steps:
filePath matches files under definitions/ that actually changed-so you do not re-publish the entire catalog on every unrelated edit. A workflow dispatch option can publish all when you need a full release or recovery.make verify) over the protobuf layout so broken protos never reach the registry.make publish): for each manifest entry, either create a new artifact or POST a new version if the artifact already exists. Versions are often branch name plus timestamp so feature branches and mainline each get traceable labels.Secrets supply REGISTRY_URL and an API key (e.g. APICURIO_API_KEY). A Node script can filter artifacts.json against CHANGED_FILES by writing filtered entries for the publish step.
A consumer keeps a smaller manifest (artifacts.json) listing only what it needs: each row has groupId, artifactId, and the local path where the downloaded file should land (mirroring definitions/ so imports stay stable). In practice a service might depend on:
Buf configuration:
buf.yaml - declares a Buf module rooted at definitions (the tree populated after fetch).buf.gen.yaml - selects remote plugins (e.g. Protocol Buffers Go, gRPC Go, and any RPC helpers) and sets go_package_prefix to the module path so generated Go matches go.mod.Topic constants:
topics.gen.json pairs each topics.json input path with a generated output path (e.g. Go files under package-aligned directories). The same Makefile pattern can emit Node, Java, or Python constants via LANG= - polyglot consumers reuse one generator.*Make targets (conceptual split across included `.mk` files):**
get-artifact-version - queries the registry for versions, optionally filtering by a branch-derived string; prefers timestamp-suffixed versions when resolving “latest for this branch”.get-artifacts-from-registry - loops the consumer manifest, resolves a version per artifact, GETs content with X-Api-Key, writes each file under definitions/. If the branch-specific version fails, fall back to a mainline version so local and CI builds stay green while schemas catch up.fetch-spec - alias for pulling all artifacts into the tree.codegen - runs buf generate over discovered *.proto files, then codegen-topics to turn each topics.gen.json mapping into Go (or other language) constants from the JSON topic lists.verify - standard Go checks: fmt, vet, golangci-lint, tests.Local workflow: make fetch-spec → make codegen → go mod tidy → build and test.
The concrete steps behind fetch and codegen differ by language: Buf and Go on one service, Maven or Gradle on another, npm or pnpm elsewhere. Make gives every repo the same shape of commands: make fetch-spec, make codegen, make verify, make publish, no matter what runs underneath. That aligns invocations across stacks so documentation, onboarding, and CI job scripts all read the same way - one vocabulary for “get contracts,” “generate code,” and “check.”
Splitting logic into included fragments (for example fetch.mk, codegen.mk, verify.mk) keeps each concern small while the top-level Makefile stays the single place developers look. Environment variables such as REGISTRY_URL and API_KEY are wired once; local runs and pipelines both call the same targets, which cuts down “works on my machine” drift. Where one recipe serves many languages (topic constants with a LANG= switch), Make is an easy place to reuse without each team re-implementing shell scripts in their own ecosystem.
Publisher CI answers: “Which definitions changed, and which registry entries get a new version?” It filters (optional), lints with Buf, then publishes to Apicurio.
Consumer CI answers: “For this commit and branch, can we fetch artifacts, generate code, and compile?” It runs make fetch-spec and make codegen, then go mod tidy, installs tooling, builds, and verifies. On the default branch only, a follow-up job may build a container image using the same fetch/codegen steps so production images match CI.
Both use the same registry URL and API key pattern; the consumer may also set BUF_TOKEN if remote Buf plugins or modules require it.
You get one place to change a message, topic list, or shared contract, automated propagation through versioned artifacts, language-appropriate codegen per service, and CI that fails when contracts or generation break-without mandating a single programming language across dozens of services. Alignment stops being a manual coordination problem and becomes pull, generate, build.
Apicurio Registry is not the only storage option, but for teams that standardize on Protobuf, JSON artifacts, and HTTP-friendly automation, it is a practical contract hub: publish from a dedicated definitions repository, consume with Make and Buf, and let each microservice stay in its native language while sharing the same bytes.
If you are designing this from scratch, invest in clear group/artifact naming, branch-aware version resolution on the consumer side (with a safe fallback), and lint-before-publish so the registry only receives definitions that already pass Buf. The payoff is fewer surprises at the wire, fewer “which copy of the proto is deployed?” incidents in production, and a single place where types, topics, and related artifacts stay authoritative while every language stack stays in sync through fetch and codegen.