Mayastor User Reference
version/2.5
version/2.5
  • Welcome to Mayastor!
  • Scope
  • Basic Architecture
  • Quickstart
    • Prerequisites
    • Preparing the Cluster
    • Deploy Mayastor
    • Configure Mayastor
      • Storage Class Parameters
    • Deploy a Test Application
  • Advanced Operations
    • Mayastor Kubectl Plugin
    • High Availability
    • Replica Rebuilds
    • Supportability
    • Monitoring
    • Node Cordon
    • Node Drain
    • Volume Snapshots
    • Volume Restore from Snapshot
  • Additional Information
    • Upgrade
      • Legacy Upgrade Support
    • Tips and Tricks
    • Performance Tips
    • I/O Path Description
    • Replica Operations
    • Call-home Metrics
    • Tested Third Party Software
    • Migrate etcd
    • Scale etcd
    • Migration for Replicated DB
      • Backup from cStor
      • Restore to Mayastor
    • Migration for Distributed DB
      • Backup from cStor
      • Restore to Mayastor
  • Platform Support
    • Mayastor Installation on MicroK8s
  • Troubleshooting
    • Basic Troubleshooting
    • Known Limitations
    • Known Issues
    • FAQs
Powered by GitBook
On this page
  • CPU isolation
  • Set Linux kernel boot parameter
  • Update grub
  • Reboot the system
  • Verify isolcpus
  • Update mayastor helm chart for CPU core specification

Was this helpful?

Export as PDF
  1. Additional Information

Performance Tips

PreviousTips and TricksNextI/O Path Description

Last updated 10 months ago

Was this helpful?

This website/page will be End-of-life (EOL) after 31 August 2024. We recommend you to visit for the latest Mayastor documentation (v2.6 and above).

Mayastor is now also referred to as OpenEBS Replicated PV Mayastor.

CPU isolation

Mayastor will fully utilize each CPU core that it was configured to run on. It will spawn a thread on each and the thread will run in an endless loop serving tasks dispatched to it without sleeping or blocking. There are also other Mayastor threads that are not bound to the CPU and those are allowed to block and sleep. However, the bound threads (also called reactors) rely on being interrupted by the kernel and other userspace processes as little as possible. Otherwise, the latency of IO may suffer.

Ideally, the only thing that interrupts Mayastor's reactor would be only kernel time-based interrupts responsible for CPU accounting. However, that is far from trivial. isolcpus option that we will be using does not prevent:

  • kernel threads and

  • other k8s pods to run on the isolated CPU

However, it prevents system services including kubelet from interfering with Mayastor.

Set Linux kernel boot parameter

Note that the best way to accomplish this step may differ, based on the Linux distro that you are using.

Add the isolcpus kernel boot parameter to GRUB_CMDLINE_LINUX_DEFAULT in the grub configuration file, with a value which identifies the CPUs to be isolated (indexing starts from zero here). The location of the configuration file to change is typically /etc/default/grub but may vary. For example when running Ubuntu 20.04 in AWS EC2 Cloud boot parameters are in /etc/default/grub.d/50-cloudimg-settings.cfg.

In the following example we assume a system with 4 CPU cores in total, and that the third and the fourth CPU cores are to be dedicated to Mayastor.

GRUB_CMDLINE_LINUX_DEFAULT="quiet splash isolcpus=2,3"

Update grub

sudo update-grub
Sourcing file `/etc/default/grub'
Sourcing file `/etc/default/grub.d/40-force-partuuid.cfg'
Sourcing file `/etc/default/grub.d/50-cloudimg-settings.cfg'
Sourcing file `/etc/default/grub.d/init-select.cfg'
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-5.8.0-29-generic
Found initrd image: /boot/microcode.cpio /boot/initrd.img-5.8.0-29-generic
Found linux image: /boot/vmlinuz-5.4.0-1037-aws
Found initrd image: /boot/microcode.cpio /boot/initrd.img-5.4.0-1037-aws
Found Ubuntu 20.04.2 LTS (20.04) on /dev/xvda1
done

Reboot the system

sudo reboot

Verify isolcpus

Basic verification is by outputting the boot parameters of the currently running kernel:

cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-5.8.0-29-generic root=PARTUUID=7213a253-01 ro console=tty1 console=ttyS0 nvme_core.io_timeout=4294967295 isolcpus=2,3 panic=-1

You can also print a list of isolated CPUs:

cat /sys/devices/system/cpu/isolated
2-3

Update mayastor helm chart for CPU core specification

To allot specific CPU cores for Mayastor's reactors, follow these steps:

  1. Execute the following command to update Mayastor's configuration. Replace <namespace> with the appropriate Kubernetes namespace where Mayastor is deployed.

kubectl mayastor upgrade -n <namespace> --set-args 'io_engine.coreList={2,3}'

In the above command, io_engine.coreList={2,3} specifies that Mayastor's reactors should operate on the third and fourth CPU cores.

The cores are numbered from 0.

Ensure that you have the Mayastor kubectl plugin installed, matching the version of your Mayastor Helm chart deployment (). You can find installation instructions in the .

OpenEBS Documentation
releases
Mayastor kubectl plugin documentation