VPC Route misconfiguration

VPC Route Misconfiguration is a chaos fault designed to simulate network disruptions across an AWS region by intentionally misconfiguring VPC route tables routes. It works by temporarily removing one or more target CIDR routes from the VPC route tables, creating a scenario where network connectivity is impacted. During the defined chaos duration, the system allows observation and monitoring of the effects caused by the misconfiguration. After the chaos period ends, the fault automatically restores the original route configuration to ensure normal operations are resumed without manual intervention.

tip

Note: It safely ignores local routes and supports multiple CIDRs via a single run.

Use cases

VPC Route Misconfiguration:

VPC Route Misconfiguration can mimics the following chaos scenarios:

Misconfigured changes to VPC route tables
Accidental deletion of external or internal routes
Loss of connectivity to critical components such as Transit Gateway (TGW), NAT Gateway (NATGW), or VPC Peering connections

Prerequisites

Kubernetes >= 1.17
Ensure you have the required AWS permissions to induce a network blackhole in the specified availability zone within the region..
Ensure that the specified VPC (if provided) includes the target availability zone.

The Kubernetes secret should have AWS access configuration (key) in the CHAOS_NAMESPACE. Below is a sample secret file.

apiVersion: v1
kind: Secret
metadata:
  name: cloud-secret
type: Opaque
stringData:
  cloud_config.yml: |-
    # Add the cloud AWS credentials respectively
    [default]
    aws_access_key_id = XXXXXXXXXXXXXXXXXXX
    aws_secret_access_key = XXXXXXXXXXXXXXX

tip

Harness CE recommends that you use the same secret name, that is, cloud-secret. Otherwise, you will need to update the AWS_SHARED_CREDENTIALS_FILE environment variable in the fault template with the new secret name and you won't be able to use the default health check probes.

Below is an example AWS policy to execute the fault.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "ec2:DescribeRouteTables",
        "ec2:DescribeVpcs",
        "ec2:DeleteRoute",
        "ec2:CreateRoute"
      ],
      "Resource": "*"
    }
  ]
}

note

Go to superset permission/policy to execute all AWS faults.
Go to common attributes and AWS-specific tunables to tune the common tunables for all faults and AWS-specific tunables.
Go to AWS named profile for chaos to use a different profile for AWS faults.

Mandatory Tunables

Tunable	Description	Notes
VPC_ID	Specify the ID of the VPC whose route tables will be targeted for connection interruption.	Example: `vpc-213214`. For more information, see VPC_ID.
TARGET_ROUTE_CIDRS	Provide the target route CIDR blocks as a comma-separated list.	Example: `10.0.0.1/12,0.0.0.0/24`. For more information, see TARGET_ROUTE_CIDRS.
TARGET_ROUTE_TABLE_IDS	Specify the IDs of the route tables associated with the given VPC. You must provide either the route table IDs or a tag.	Example: `rtb-123,rtb-456`. For more information, see TARGET_ROUTE_TABLE_IDS.
TARGET_ROUTE_TABLE_TAG	Provide the tag key-value pair used to identify the target route tables within the VPC. It is in the format `key=value` (for example, 'team=devops'). You must provide either the route table IDs or a tag.	Example: `type=chaos`. For more information, see TARGET_ROUTE_TABLE_TAG.
REGION	Specify the AWS Region where the target VPC and route tables reside.	Example: `us-east-1`.

Optional Tunables

Tunable	Description	Notes
TOTAL_CHAOS_DURATION	Duration to insert chaos (in seconds).	Default: 30 s. For more information, go to duration of the chaos.
CHAOS_INTERVAL	Interval between successive instance terminations (in seconds).	Default: 30s. For more information, go to chaos interval.
AWS_SHARED_CREDENTIALS_FILE	Path to the AWS secret credentials.	Default: `/tmp/cloud_config.yml`.
CHAOS_INTERVAL	Duration between the attachment and detachment of the volumes (in seconds).	Default: 30 s. For more information, go to chaos interval.
SEQUENCE	Sequence of chaos execution for multiple volumes.	Default: parallel. Supports serial and parallel. For more information, go to sequence of chaos execution.
RAMP_TIME	Period to wait before and after injecting chaos (in seconds).	For example, 30 s. For more information, go to ramp time.

VPC ID

Specify the ID of the VPC whose route tables will be targeted for connection interruption. Tune it by using the VPC_ID environment variable.

The following YAML snippet illustrates the use of this environment variable:

# contains vpc ids for given region
apiVersion: litmuschaos.io/v1alpha1
kind: ChaosEngine
metadata:
  name: engine-nginx
spec:
  engineState: "active"
  chaosServiceAccount: litmus-admin
  experiments:
  - name: vpc-route-misconfiguration
    spec:
      components:
        env:
        # target vpc id for the chaos
        - name: VPC_ID
          value: 'vpc-21312481928410'
        - name: TARGET_ROUTE_CIDRS
          value: '0.0.0.0/0'

Target route CIDRs

Comma-separated list of the target routes CIDR under for the given route tables. Tune it by using the TARGET_ROUTE_CIDRS environment variable.

The following YAML snippet illustrates the use of this environment variable:

# contains route table cidr for given region
apiVersion: litmuschaos.io/v1alpha1
kind: ChaosEngine
metadata:
  name: engine-nginx
spec:
  engineState: "active"
  chaosServiceAccount: litmus-admin
  experiments:
  - name: vpc-route-misconfiguration
    spec:
      components:
        env:
        # target route table cidr for the chaos
        - name: TARGET_ROUTE_CIDRS
          value: '10.1.0.0/16,0.0.0.0/0'
        - name: TARGET_ROUTE_TABLE_IDS
          value: 'rtb-212'
        - name: VPC_ID
          value: 'vpc-21312'

Route Table IDs

Comma-separated list of the target route table Ids that are associated with the given VPC. Tune it by using the TARGET_ROUTE_TABLE_IDS environment variable.

The following YAML snippet illustrates the use of this environment variable:

# contains route tables by id for given region
apiVersion: litmuschaos.io/v1alpha1
kind: ChaosEngine
metadata:
  name: engine-nginx
spec:
  engineState: "active"
  chaosServiceAccount: litmus-admin
  experiments:
  - name: vpc-route-misconfiguration
    spec:
      components:
        env:
        # target route tables by id for the chaos
        - name: TARGET_ROUTE_TABLE_IDS
          value: 'rtb-23213'
        - name: VPC_ID
          value: 'vpc-21312'

Route Table Tag

Specify the tag to filter the target route tables associated with the given VPC. Tune it by using the TARGET_ROUTE_TABLE_TAG environment variable.

The following YAML snippet illustrates the use of this environment variable:

# contains route tables by tag for given region
apiVersion: litmuschaos.io/v1alpha1
kind: ChaosEngine
metadata:
  name: engine-nginx
spec:
  engineState: "active"
  chaosServiceAccount: litmus-admin
  experiments:
  - name: vpc-route-misconfiguration
    spec:
      components:
        env:
        # target route tables by tag for the chaos
        - name: TARGET_ROUTE_TABLE_TAG
          value: 'type=chaos'
        - name: VPC_ID
          value: 'vpc-21312'

Use cases​

Prerequisites​

Mandatory Tunables​

Optional Tunables​

VPC ID​

Target route CIDRs​

Route Table IDs​

Route Table Tag​

Use cases

Prerequisites

Mandatory Tunables

Optional Tunables

VPC ID

Target route CIDRs

Route Table IDs

Route Table Tag