Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[crmsh-4.6] Dev: pre-migration: implement pre-migration checks for SLES 16 (jsc#PED-11808) #1629

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
4488385
Dev: ui_cluster: add 'crm cluster health sles [--fix]' (jsc#PED-8252)
nicholasyang2022 Oct 23, 2024
b2265ef
Dev: pre-migration: preserve only checks for SLES 16 (jsc#PED-8252)
nicholasyang2022 Dec 10, 2024
49779b4
Dev: migration: run checks on remote nodes (jsc#PED-8252)
nicholasyang2022 Nov 1, 2024
689b246
backport to python 3.6
nicholasyang2022 Dec 13, 2024
1607318
Dev: pre-migration: add checks for used corosync features (jsc#PED-11…
nicholasyang2022 Dec 13, 2024
26869fd
Dev: pre-migration: add service status checks (jsc#PED-11808)
nicholasyang2022 Dec 13, 2024
25e0dff
Dev: pre-migration: add checks for package version (jsc#PED-11808)
nicholasyang2022 Dec 13, 2024
f061b4f
Dev: pre-migration: check SAPHanaSR Classic resource agents (jsc#PED-…
nicholasyang2022 Dec 31, 2024
a38ec9a
Dev: pre-migration: check removed resource agents (jsc#PED-11808)
nicholasyang2022 Dec 31, 2024
440ea0e
Dev: pre-migration: check removed fence agents (jsc#PED-11808)
nicholasyang2022 Dec 31, 2024
bbfcfa4
Dev: pre-migration: check if ocfs2 is used (jsc#PED-11808)
nicholasyang2022 Jan 1, 2025
36f7d23
Dev: add pre-migration checks for pacemaker version (jsc#PED-11808)
nicholasyang2022 Jan 2, 2025
52dac4d
Dev: pre-migration: better PASS/FAIL messages (jsc#PED-11808)
nicholasyang2022 Jan 6, 2025
14f5b2e
Dev: pre-migration: do not colorize hostname header (jsc#PED-11808)
nicholasyang2022 Jan 6, 2025
4dae2e3
Dev: pre-migration: add completer for 'crm cluster health sles16' (js…
nicholasyang2022 Jan 6, 2025
22fd915
Fix: ui_cluster: fix do_health usage output
nicholasyang2022 Jan 6, 2025
abbbc3e
Dev: pre-migration: add message about removing stonith:external/sbd (…
nicholasyang2022 Jan 7, 2025
843d637
Dev: pre-migration: refine the message for stopped cluster services (…
nicholasyang2022 Jan 7, 2025
d8b62b1
Doc: ui_cluster: add document for `crm cluster health hawk2|sles16`
nicholasyang2022 Jan 7, 2025
2368846
Fix: cibquery: grouped primitives is missing from query results (jsc#…
nicholasyang2022 Jan 9, 2025
b6b8987
Dev: behave: add functional tests for pre-migration checks (jsc#PED-1…
nicholasyang2022 Jan 10, 2025
26ccee4
Refactor: cibquery: has_primitive_filesystem_ocfs2 to has_primitive_f…
nicholasyang2022 Jan 13, 2025
252befd
Dev: pre-migration: add summary section to output (jsc#PED-11808)
nicholasyang2022 Jan 13, 2025
7670957
Dev: migration: refine message wording (jsc#PED-11808)
nicholasyang2022 Jan 16, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions .github/workflows/crmsh-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -449,6 +449,21 @@ jobs:
token: ${{ secrets.CODECOV_TOKEN }}
flags: integration

functional_test_migration:
runs-on: ubuntu-24.04
timeout-minutes: 40
steps:
- uses: actions/checkout@v4
- name: functional test for migration
run: |
echo '{ "exec-opts": ["native.cgroupdriver=systemd"] }' | sudo tee /etc/docker/daemon.json
sudo systemctl restart docker.service
$DOCKER_SCRIPT `$GET_INDEX_OF migration` && $DOCKER_SCRIPT -d && $DOCKER_SCRIPT -u `$GET_INDEX_OF migration`
- uses: codecov/codecov-action@v4
with:
token: ${{ secrets.CODECOV_TOKEN }}
flags: integration

original_regression_test:
runs-on: ubuntu-20.04
timeout-minutes: 40
Expand Down
1 change: 1 addition & 0 deletions MANIFEST.in
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
recursive-include crmsh *.txt
25 changes: 25 additions & 0 deletions crmsh/cibquery.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
"""utilities for parsing CIB xml"""
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it's better to move these codes into xmlutil.py instead of creating a new py file?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No. xmlutil.py is too long, and its name does not reflect its content. Instead of general xml processing utilities, it is a mixed mess with shell utils, CIB specific routines, and even CLI handlers.

import collections
import typing

import lxml.etree


ResourceAgent = collections.namedtuple('ResourceAgentDO', ['m_class', 'm_provider', 'm_type'])
# class: str
# provider: Optional[str]
# type: str


def get_configured_resource_agents(cib: lxml.etree.Element) -> typing.Set[ResourceAgent]:
return set(
ResourceAgent(e.get('class'), e.get('provider'), e.get('type'))
for e in cib.xpath('/cib/configuration/resources//primitive')
)


def has_primitive_filesystem_with_fstype(cib: lxml.etree.Element, fstype: str) -> bool:
return bool(cib.xpath(
'/cib/configuration/resources//primitive[@class="ocf" and @provider="heartbeat" and @type="Filesystem"]'
f'/instance_attributes/nvpair[@name="fstype" and @value="{fstype}"]'
))
147 changes: 147 additions & 0 deletions crmsh/migration-supported-resource-agents.txt
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about moving this file under crmsh/crmsh/data, in case there are other txt files in the future?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer not to do that. I think this will need to add a python module crmsh.data, making our build scripts more complex.

Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
ocf:heartbeat:CTDB
ocf:heartbeat:ClusterMon
ocf:heartbeat:Delay
ocf:heartbeat:Dummy
ocf:heartbeat:Filesystem
ocf:heartbeat:IPaddr2
ocf:heartbeat:IPsrcaddr
ocf:heartbeat:IPv6addr
ocf:heartbeat:LVM-activate
ocf:heartbeat:MailTo
ocf:heartbeat:NodeUtilization
ocf:heartbeat:Raid1
ocf:heartbeat:Route
ocf:heartbeat:SAPDatabase
ocf:heartbeat:SAPInstance
ocf:heartbeat:SendArp
ocf:heartbeat:Squid
ocf:heartbeat:Stateful
ocf:heartbeat:VirtualDomain
ocf:heartbeat:WAS
ocf:heartbeat:WAS6
ocf:heartbeat:Xinetd
ocf:heartbeat:aliyun-vpc-move-ip
ocf:heartbeat:apache
ocf:heartbeat:aws-vpc-move-ip
ocf:heartbeat:aws-vpc-route53
ocf:heartbeat:awseip
ocf:heartbeat:awsvip
ocf:heartbeat:azure-events
ocf:heartbeat:azure-events-az
ocf:heartbeat:azure-lb
ocf:heartbeat:conntrackd
ocf:heartbeat:corosync-qnetd
ocf:heartbeat:crypt
ocf:heartbeat:db2
ocf:heartbeat:dhcpd
ocf:heartbeat:docker
ocf:heartbeat:docker-compose
ocf:heartbeat:dummypy
ocf:heartbeat:ethmonitor
ocf:heartbeat:exportfs
ocf:heartbeat:galera
ocf:heartbeat:garbd
ocf:heartbeat:gcp-ilb
ocf:heartbeat:gcp-pd-move
ocf:heartbeat:gcp-vpc-move-ip
ocf:heartbeat:gcp-vpc-move-vip
ocf:heartbeat:iSCSILogicalUnit
ocf:heartbeat:iSCSITarget
ocf:heartbeat:iface-bridge
ocf:heartbeat:iface-macvlan
ocf:heartbeat:iface-vlan
ocf:heartbeat:ldirectord
ocf:heartbeat:lvmlockd
ocf:heartbeat:mariadb
ocf:heartbeat:mdraid
ocf:heartbeat:mpathpersist
ocf:heartbeat:mysql
ocf:heartbeat:mysql-proxy
ocf:heartbeat:named
ocf:heartbeat:nfsnotify
ocf:heartbeat:nfsserver
ocf:heartbeat:nginx
ocf:heartbeat:nvmet-namespace
ocf:heartbeat:nvmet-port
ocf:heartbeat:nvmet-subsystem
ocf:heartbeat:oraasm
ocf:heartbeat:oracle
ocf:heartbeat:oralsnr
ocf:heartbeat:osceip
ocf:heartbeat:ovsmonitor
ocf:heartbeat:pgagent
ocf:heartbeat:pgsql
ocf:heartbeat:podman
ocf:heartbeat:portblock
ocf:heartbeat:postfix
ocf:heartbeat:powervs-subnet
ocf:heartbeat:rabbitmq-cluster
ocf:heartbeat:rabbitmq-server-ha
ocf:heartbeat:redis
ocf:heartbeat:rsyncd
ocf:heartbeat:sfex
ocf:heartbeat:sg_persist
ocf:heartbeat:slapd
ocf:heartbeat:storage-mon
ocf:heartbeat:symlink
ocf:heartbeat:tomcat
ocf:suse:aws-vpc-move-ip
ocf:suse:SAPHanaController
ocf:suse:SAPHanaFilesystem
ocf:suse:SAPHanaTopology
stonith:fence_aliyun
stonith:fence_alom
stonith:fence_apc
stonith:fence_apc-snmp
stonith:fence_aws
stonith:fence_azure-arm
stonith:fence_bladecenter
stonith:fence_brocade
stonith:fence_cisco-mds
stonith:fence_cisco-ucs
stonith:fence_compute
stonith:fence_docker
stonith:fence_drac5
stonith:fence_eaton-snmp
stonith:fence_eaton-ssh
stonith:fence_emerson
stonith:fence_eps
stonith:fence_gce
stonith:fence_hds-cb
stonith:fence_hpblade
stonith:fence_ibm-powervs
stonith:fence_ibm-vpc
stonith:fence_ibmblade
stonith:fence_ibmz
stonith:fence_ifmib
stonith:fence_ilo-moonshot
stonith:fence_ilo-mp
stonith:fence_ilo-ssh
stonith:fence_ilo2
stonith:fence_intelmodular
stonith:fence_ipdu
stonith:fence_ipmilan
stonith:fence_ironic
stonith:fence_kdump
stonith:fence_ldom
stonith:fence_lpar
stonith:fence_mpath
stonith:fence_netio
stonith:fence_openstack
stonith:fence_pve
stonith:fence_raritan
stonith:fence_rcd-serial
stonith:fence_redfish
stonith:fence_rhevm
stonith:fence_rsa
stonith:fence_rsb
stonith:fence_sanbox2
stonith:fence_sbd
stonith:fence_scsi
stonith:fence_vbox
stonith:fence_virsh
stonith:fence_vmware
stonith:fence_vmware-rest
stonith:fence_wti
stonith:fence_xenapi
stonith:fence_zvm
Copy link
Collaborator

@liangxin1300 liangxin1300 Jan 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How has the above data been generated?
Can we automatically do that based on the locally installed resource-agents and fence-agents-* rpm?
Or, can we generate the data at the time crmsh using it (_load_supported_resource_agents)?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How has the above data been generated?

Extracted the file list from resource-agents/fence-agents.

Can we automatically do that based on the locally installed resource-agents and fence-agents-* rpm?

No, because we need to know what is supported by SLES 16 instead of SLES 15 SP6.

Loading