This article aims to introduce GoFrame framework's support for microservices-mono-repo management mode, guiding developers on how to conduct code development and collaboration in a microservices-mono-repo management mode.

1. Pre-reading

Before starting this chapter, it is recommended to first understand the basic concepts and respective pros and cons of monolith repositories (monolith), microservices-multi-repo management (multi-repo), and microservices-mono-repo management (mono-repo): What are the advantages and disadvantages of monolith and multi-repo, and which solution is better for microservices?

Management constraints of code repositories do not belong to the framework’s responsibilities. The GoFrame framework's scaffold itself also supports commands to initialize two kinds of repository projects - single repository (mono-repo) and multiple repositories (monolith/multi-repo), to meet the needs of different teams. The specific choice of code repository management mode is decided by the development team based on their own needs, scenarios, and habits.

tip

To simplify and clarify the description of microservices-mono-repo management (mono-repo), we will refer to it as large repo management moving forward.

2. Large Repo Management

1. Division of Repository Responsibilities

As noted in the pre-reading article, there's no silver bullet in the world, and large repos have both advantages and disadvantages. The most apparent drawbacks are permission control and repository bloating. To better manage the code repository and avoid the higher costs associated with these two drawbacks, we recommend minimizing the scope of microservices in a large repository as much as possible. The decision of which microservices need to be maintained in the repository depends on the frequency of cooperation between services.

1) When intra-team cooperation frequency is higher than inter-team

• A typical scenario is for non-microservice architecture products, where service management responsibilities can be divided according to each business team. This way, the team can maintain scattered services in a unified code repository, fully utilizing the advantages of large repo management to improve intra-team development and maintenance efficiency.
• Another scenario is when the number of business microservices is not high (for example, within 50), merging them into a large repository for management is feasible. Note that the number of services in large repo management is not determined by the number of people in the organizational structure.

2) When cooperation frequency is high between multiple teams

If the number of business microservices is high, and interactions and collaborations between services are frequent, merging these services into a large repository for management can significantly improve collaboration efficiency. This typically occurs when microservices are within the same product line, across teams but not across centers or departments. Since this involves collaboration across multiple teams, it requires certain managerial authorization to drive.

tip

Microservice management is not only about code organization but also about organizational management.

2. How Microservices Collaborate in a Large Repo

1) Management of Code Visibility

The only thing that can be exposed between services is the interface, namely the API. The internal logic of each service should not be visible externally. Go language has a beneficial internal feature which can satisfy visibility management requirements. As shown in the large repo code example below, several services are managed under the app directory, each exposing its own api directory for direct reference by other services (enhancing collaboration efficiency), but the internal business logic is contained in the internal directory, not visible (and therefore inaccessible) to other services.

2) Service Interface Invocation

Protocol files should be maintained separately in each service's directory. If the protocol file requires compilation, the compiled file should also be stored in its own service directory. The caller does not need to recompile and manage the target service's protocol files separately. For instance, with HTTP API interface definitions, the caller can directly reference the target service's API interface definition (in the following screenshot, khaos-shark is the caller, khaos-oss is the service provider).

The same logic applies to RPC interface calls between microservices (in the following screenshot, user-api is the caller, user-rpc is the service provider).

3) Strict Compatibility Requirements

As introduced above, through large repo code management, all services within the large repo maintain consistent versions. When the service API relied upon is updated, the caller's service (using the SDK) will also automatically get updated. This requires all services within the repository to strictly ensure interface compatibility, otherwise, there may be issues with interface invocation: at best, the caller's service compilation fails requiring code adjustments, at worst, it compiles successfully but throws runtime errors affecting the business. Publicly shared large repo base components will also be affected by compatibility issues.

Key points for ensuring compatibility in code design:

• Do not arbitrarily delete or modify interface parameters, parameter names, types, or validation logic.
• When an interface must undergo non-compatible updates, use interface versioning management (such as v1, v2, v3...).
• Public components should rely on stable and mature external components as much as possible. If a custom component is necessary, ensure the compatibility of exposed methods. For example: basic functions like json.Marshal&Unmarshal may be wrapped by some libraries/functions, but later users may not know or trust this function, leading to redundant rewrites. Over time, these libraries/functions become unmaintained.

3. Microservice Containerization Support under Large Repos

1) Unified Image Repository Management

Scattered image repositories can reduce efficiency in service containerization management and maintenance. To facilitate unified service containerization management, we recommend using a unified image repository for services under a large repo. The image repository address is maintained in the tool configuration files for each service:

2) Unified Compile, Submit Commands

The framework provides commonly used commands to compile programs, build images, and submit images.

• make build

Compiles the program to generate binary files.

For more information, please refer to the documentation: Cross-Compiling

• make image

Compiles the program and builds the image, generating a Docker image.

Use make image TAG=xxx to specify the tag name of the compiled image.

For more information, please refer to the documentation: Image Building

• make image.push

Compiles the program, builds the image, and pushes it to the configured image repository.

Use make image.push TAG=xxx to specify the tag name of the compiled image.

3) Unified Deployment, Debugging Commands

The framework offers common commands for containerized deployment of Kubernetes clusters, as well as integrated compile-deploy development commands.

• make deploy TAG=xxx

Deploys the current service to a kubernetes cluster connected via local kubeconfig, where TAG is used to specify the overlays directory under the deploy directory. Deployment yaml file management uses the industry-standard kustomize tool. For detailed documentation, please refer to: https://kubernetes.io/zh-cn/docs/tasks/manage-kubernetes-objects/kustomization/

• make image.push deploy TAG=xxx

This command is a development debugging directive used for compiling binaries, building and pushing Docker images, deploying Kubernetes applications, and restarting the application with a single command.

4. Other Framework Commands under Large Repos

The framework provides a wealth of tool command support for project engineering management. These commands often need to be executed in specific service directories, such as ./app/service-name

1) make cli

Used to upgrade the local framework CLI to the latest stable version.

2) make up

Used to upgrade the local framework to the latest stable version in the community.

For more information, please refer to the documentation: Version Upgrade

3) make dao

Used to generate DAO/Entity/DO code files.

For more information, please refer to the documentation: Dao/Do/Entity Generating

4) make service

Parses the logic directory and automatically generates internal call interfaces. In Goland IDE, this command is often used in conjunction with an automated Watcher file change to auto-generate content; see the official documentation for details.

For more information, please refer to the documentation: Service Generating

5) make enums

Used to parse specified code directories (default is api directory) and auto-generate enums load code.

For more information, please refer to the documentation: Enums Maintenance

6) More Commands

For more command support, please refer to the framework's official tools introduction section: CLI Tool