Add a New Chaos Experiment Type
This document describes how to add a new chaos experiment type.
The following walks you through an example of HelloWorldChaos
, a new chaos experiment type that prints Hello world!
to the log. The steps include:
- Step 1: Define the schema of HelloWorldChaos
- Step 2: Register the CRD
- Step 3: Register the event handler for helloworld objects
- Step 4: Build Docker images
- Step 5: Run HelloWorldChaos
Step 1: Define the schema of HelloWorldChaos
-
Add
helloworldchaos_types.go
to theapi/v1alpha1
API directory with the following content:package v1alpha1
import (
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
// +kubebuilder:object:root=true
// +chaos-mesh:experiment
// +chaos-mesh:oneshot=true
// HelloWorldChaos is the Schema for the helloworldchaos API
type HelloWorldChaos struct {
metav1.TypeMeta `json:",inline"`
metav1.ObjectMeta `json:"metadata,omitempty"`
Spec HelloWorldChaosSpec `json:"spec"`
Status HelloWorldChaosStatus `json:"status,omitempty"`
}
// HelloWorldChaosSpec defines the desired state of HelloWorldChaos
type HelloWorldChaosSpec struct {
// ContainerSelector specifies the target for injection
ContainerSelector `json:",inline"`
// Duration represents the duration of the chaos
// +optional
Duration *string `json:"duration,omitempty"`
// RemoteCluster represents the remote cluster where the chaos will be deployed
// +optional
RemoteCluster string `json:"remoteCluster,omitempty"`
}
// HelloWorldChaosStatus defines the observed state of HelloWorldChaos
type HelloWorldChaosStatus struct {
ChaosStatus `json:",inline"`
}
// GetSelectorSpecs is a getter for selectors
func (obj *HelloWorldChaos) GetSelectorSpecs() map[string]interface{} {
return map[string]interface{}{
".": &obj.Spec.ContainerSelector,
}
}This file defines the schema type of
HelloWorldChaos
, which can be described in a YAML file:apiVersion: chaos-mesh.org/v1alpha1
kind: HelloWorldChaos
metadata:
name: <resource name>
namespace: <namespace>
spec:
duration: <duration>
#...
Step 2: Register the CRD
You need to register the CRD (Custom Resource Definition) of HelloWorldChaos
to interact it with Kubernetes API.
-
To combine the CRD into manifests/crd.yaml, append
config/crd/bases/chaos-mesh.org_helloworldchaos.yaml
we generated in the previous step toconfig/crd/kustomization.yaml
:resources:
- bases/chaos-mesh.org_podchaos.yaml
- bases/chaos-mesh.org_networkchaos.yaml
- bases/chaos-mesh.org_iochaos.yaml
- bases/chaos-mesh.org_helloworldchaos.yaml # This is the new line -
Run
make generate
in the root directory of Chaos Mesh, which generates a boilerplate ofHelloWorldChaos
for Chaos Mesh to compile:make generate
Then you can see the definition of
HelloWorldChaos
inmanifests/crd.yaml
.
Step 3: Register the event handler for helloworldchaos objects
-
Create a new file
controllers/chaosimpl/helloworldchaos/types.go
with the following content:package helloworldchaos
import (
"context"
"github.com/go-logr/logr"
"go.uber.org/fx"
"sigs.k8s.io/controller-runtime/pkg/client"
"github.com/chaos-mesh/chaos-mesh/api/v1alpha1"
impltypes "github.com/chaos-mesh/chaos-mesh/controllers/chaosimpl/types"
"github.com/chaos-mesh/chaos-mesh/controllers/chaosimpl/utils"
)
var _ impltypes.ChaosImpl = (*Impl)(nil)
type Impl struct {
client.Client
Log logr.Logger
decoder *utils.ContainerRecordDecoder
}
// This corresponds to the Apply phase of HelloWorldChaos. The execution of HelloWorldChaos will be triggered.
func (impl *Impl) Apply(ctx context.Context, index int, records []*v1alpha1.Record, obj v1alpha1.InnerObject) (v1alpha1.Phase, error) {
impl.Log.Info("Hello world!")
return v1alpha1.Injected, nil
}
// This corresponds to the Recover phase of HelloWorldChaos. The reconciler will be triggered to recover the chaos action.
func (impl *Impl) Recover(ctx context.Context, index int, records []*v1alpha1.Record, obj v1alpha1.InnerObject) (v1alpha1.Phase, error) {
impl.Log.Info("Goodbye world!")
return v1alpha1.NotInjected, nil
}
// NewImpl returns a new HelloWorldChaos implementation instance.
func NewImpl(c client.Client, log logr.Logger, decoder *utils.ContainerRecordDecoder) *impltypes.ChaosImplPair {
return &impltypes.ChaosImplPair{
Name: "helloworldchaos",
Object: &v1alpha1.HelloWorldChaos{},
Impl: &Impl{
Client: c,
Log: log.WithName("helloworldchaos"),
decoder: decoder,
},
ObjectList: &v1alpha1.HelloWorldChaosList{},
}
}
var Module = fx.Provide(
fx.Annotated{
Group: "impl",
Target: NewImpl,
},
) -
Chaos Mesh uses the fx library for dependency injection. To register
HelloWorldChaos
in the controller manager, add a line tocontrollers/chaosimpl/fx.go
:var AllImpl = fx.Options(
gcpchaos.Module,
stresschaos.Module,
jvmchaos.Module,
timechaos.Module,
helloworldchaos.Module // Add a new line. Make sure you have imported helloworldchaos first.
//...
)Then in
controllers/types/types.go
, append the following content intoChaosObjects
:var ChaosObjects = fx.Supply(
//...
fx.Annotated{
Group: "objs",
Target: Object{
Name: "helloworldchaos",
Object: &v1alpha1.HelloWorldChaos{},
},
},
)
Step 4: Build Docker images
-
Build the production images:
make image
-
If you deploy the Kubernetes cluster using minikube, then you need to load images into the cluster:
minikube image load ghcr.io/chaos-mesh/chaos-dashboard:latest
minikube image load ghcr.io/chaos-mesh/chaos-mesh:latest
minikube image load ghcr.io/chaos-mesh/chaos-daemon:latest
Step 5: Run HelloWorldChaos
In this step, you need to deploy Chaos Mesh with your latest changes to test HelloWorldChaos.
-
Register the CRD in your cluster:
kubectl create -f manifests/crd.yaml
You can see
HelloWorldChaos
is created from the output:customresourcedefinition.apiextensions.k8s.io/helloworldchaos.chaos-mesh.org created
Now you can get the CRD of
HelloWorldChaos
using the command below:kubectl get crd helloworldchaos.chaos-mesh.org
-
Deploy Chaos Mesh:
helm install chaos-mesh helm/chaos-mesh -n=chaos-mesh --set controllerManager.leaderElection.enabled=false,dashboard.securityMode=false
To verify the deployment is successful, you can check all Pods in the
chaos-mesh
namespace:kubectl get pods --namespace chaos-mesh -l app.kubernetes.io/instance=chaos-mesh
-
Deploy a deployment for testing, we can use an example echo server from minikube docs:
kubectl create deployment hello-minikube --image=kicbase/echo-server:1.0
Wait to see the pod is running:
kubectl get pods
Example output:
NAME READY STATUS RESTARTS AGE
hello-minikube-77b6f68484-dg4sw 1/1 Running 0 2m -
Create a
hello.yaml
file with the following content:apiVersion: chaos-mesh.org/v1alpha1
kind: HelloWorldChaos
metadata:
name: hello-world
namespace: chaos-mesh
spec:
selector:
labelSelectors:
app: hello-minikube
mode: one
duration: 1h -
Run:
kubectl apply -f hello.yaml
# helloworldchaos.chaos-mesh.org/hello-world createdNow you can check if
chaos-controller-manager
hasHello world!
in its logs:kubectl logs -n chaos-mesh chaos-controller-manager-xxx
Example output:
2023-07-16T06:19:40.068Z INFO records records/controller.go:149 apply chaos {"id": "default/hello-minikube-77b6f68484-dg4sw/echo-server"}
2023-07-16T06:19:40.068Z INFO helloworldchaos helloworldchaos/types.go:26 Hello world!
What's Next
If you encounter any problems during the process, create an issue in the Chaos Mesh repository.
In the next section, we'll learn more about how to extend the behavior of HelloWorldChaos
.