kubernetes: Deploy kubernetes-provider when creating a cluster by davidjumani · Pull Request #5254 · apache/cloudstack

davidjumani · 2021-07-29T10:49:56Z

Description

This PR automatically deploys the cloudstack-kubernetes-provider when deploying a kubernetes cluster

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale

Major
Minor

How Has This Been Tested?

kubectl get pods -A

NAMESPACE              NAME                                         READY   STATUS    RESTARTS   AGE
kube-system            cloud-controller-manager-66d6576549-rd5zf    1/1     Running   0          44m
.......

davidjumani · 2021-07-29T10:55:26Z

@blueorangutan package

blueorangutan · 2021-07-29T10:56:03Z

@davidjumani a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2021-07-29T11:46:42Z

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian. SL-JID 689

blueorangutan · 2021-07-30T00:16:15Z

Trillian test result (tid-1413)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 39494 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5254-t1413-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Smoke tests completed. 88 look OK, 1 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_01_invalid_upgrade_kubernetes_cluster	`Failure`	0.07	test_kubernetes_clusters.py
test_02_deploy_and_upgrade_kubernetes_cluster	`Failure`	0.04	test_kubernetes_clusters.py
test_03_deploy_and_scale_kubernetes_cluster	`Failure`	0.04	test_kubernetes_clusters.py
test_04_basic_lifecycle_kubernetes_cluster	`Failure`	0.05	test_kubernetes_clusters.py
test_05_delete_kubernetes_cluster	`Failure`	0.05	test_kubernetes_clusters.py
test_07_deploy_kubernetes_ha_cluster	`Failure`	0.04	test_kubernetes_clusters.py
test_08_deploy_and_upgrade_kubernetes_ha_cluster	`Failure`	0.04	test_kubernetes_clusters.py
test_09_delete_kubernetes_ha_cluster	`Failure`	0.04	test_kubernetes_clusters.py

yadvr · 2021-07-30T05:26:15Z

We've another cks PR @davidjumani which one should be reviewed/tested/merged first cc @Pearl1594 or is this result of testing and your RM work on k8s provider?

davidjumani · 2021-07-30T05:41:03Z

@rhtyd They're both independent, and this is related to the RM for the k8s provider

davidjumani · 2021-07-30T08:22:31Z

@blueorangutan package

blueorangutan · 2021-07-30T08:23:02Z

@davidjumani a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2021-07-30T08:48:12Z

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian. SL-JID 708

shwstppr

@davidjumani couple of minor comments and a query on state transition

davidjumani · 2021-07-30T09:48:52Z

@blueorangutan package

blueorangutan · 2021-07-30T10:56:56Z

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian. SL-JID 709

davidjumani · 2021-07-30T11:07:32Z

@blueorangutan test

blueorangutan · 2021-07-30T11:08:03Z

@davidjumani a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2021-07-30T22:03:42Z

Trillian test result (tid-1431)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 37602 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5254-t1431-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_outofbandmanagement_nestedplugin.py
Smoke tests completed. 88 look OK, 1 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_02_deploy_and_upgrade_kubernetes_cluster	`Failure`	500.36	test_kubernetes_clusters.py

davidjumani · 2021-07-31T05:09:25Z

@blueorangutan test

blueorangutan · 2021-07-31T05:10:03Z

@davidjumani a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

…d/kubernetes/cluster/actionworkers/KubernetesClusterActionWorker.java Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

davidjumani · 2021-08-02T05:50:32Z

@blueorangutan package

blueorangutan · 2021-08-02T05:51:03Z

@davidjumani a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2021-08-02T06:13:11Z

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian. SL-JID 722

davidjumani · 2021-08-02T06:16:10Z

@blueorangutan test

blueorangutan · 2021-08-02T06:17:03Z

@davidjumani a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2021-08-02T18:17:41Z

Trillian test result (tid-1445)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 41561 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5254-t1445-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_projects.py
Intermittent failure detected: /marvin/tests/smoke/test_routers_network_ops.py
Intermittent failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_router_nics.py
Smoke tests completed. 89 look OK, 0 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File

yadvr · 2021-08-03T05:43:37Z

+    private void validateEndpointUrl() {
+        String csUrl = ApiServiceConfiguration.ApiServletPath.value();
+        if (csUrl == null || csUrl.contains("localhost")) {
+            throw new InvalidParameterValueException("Global setting endpointe.url has to be set to the Management Server's API end point");


You may want to fix the typo?

Nope, it's a global variable

What about ApiServiceConfiguration.ApiServletPath.key() instead of endpointe.url?

Would be better to retrieve as @nvazquez said. Thus, if/when this gets fixed it will automatically reflect in this message.

yadvr · 2021-08-03T05:44:17Z

+        if (owner == null) {
+            owner = CallContext.current().getCallingAccount();
+        }
+        String username = owner.getAccountName() + "-" + KUBEADMIN_ACCOUNT_NAME;


Do we assume/create a service account/user globally, or is it per account now (a service user just for k8s use-case)?

Nevermind read the code, looks like we create if it doesn't exist.

Per account basis just for kube

yadvr · 2021-08-03T05:45:24Z

+            cls.endpoint_url = Configurations.list(cls.apiclient, name="endpointe.url")[0].value
+            if "localhost" in cls.endpoint_url:
+                endpoint_url = "http://%s:%d/client/api " %(cls.mgtSvrDetails["mgtSvrIp"], cls.mgtSvrDetails["port"])
+                cls.debug("Setting endpointe.url to %s" %(endpoint_url))


Is endpointe a thing or typo?

Nope, it's a global variable

yadvr

LGTM - some testing may be needed and it may conflict #4329

nvazquez · 2021-08-03T12:15:41Z

@blueorangutan package

blueorangutan · 2021-08-03T12:16:02Z

@nvazquez a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2021-08-03T12:39:11Z

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian. SL-JID 736

GabrielBrascher · 2021-08-03T19:11:25Z

+    private void validateEndpointUrl() {
+        String csUrl = ApiServiceConfiguration.ApiServletPath.value();
+        if (csUrl == null || csUrl.contains("localhost")) {
+            throw new InvalidParameterValueException("Global setting endpointe.url has to be set to the Management Server's API end point");


Would be better to retrieve as @nvazquez said. Thus, if/when this gets fixed it will automatically reflect in this message.

nvazquez

LGTM, simply left minor suggestions, thanks @davidjumani

davidjumani · 2021-08-05T05:01:40Z

@blueorangutan package

blueorangutan · 2021-08-05T05:02:03Z

@davidjumani a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2021-08-05T05:28:16Z

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian. SL-JID 756

davidjumani · 2021-08-05T07:39:37Z

@blueorangutan test

blueorangutan · 2021-08-05T07:40:02Z

@davidjumani a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2021-08-05T22:09:33Z

Trillian test result (tid-1485)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 50493 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5254-t1485-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_privategw_acl.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Smoke tests completed. 87 look OK, 2 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_03_vpc_privategw_restart_vpc_cleanup	`Failure`	372.75	test_privategw_acl.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL	`Failure`	549.85	test_vpc_redundant.py
test_03_create_redundant_VPC_1tier_2VMs_2IPs_2PF_ACL_reboot_routers	`Failure`	478.00	test_vpc_redundant.py
test_05_rvpc_multi_tiers	`Failure`	529.28	test_vpc_redundant.py

yadvr · 2021-08-09T09:14:59Z

Merging this based on regression tests and lgtms, further review/testing and improvements can be continued at #4329

kubernetes: Deploy kubernetes-provider when creating a cluster

961fdd2

davidjumani force-pushed the add-kube-provider branch from 59433d4 to 961fdd2 Compare July 29, 2021 10:54

davidjumani requested a review from shwstppr July 29, 2021 10:56

davidjumani added component:kubernetes type:enhancement labels Jul 29, 2021

davidjumani added this to the 4.16.0.0 milestone Jul 29, 2021

yadvr requested a review from Pearl1594 July 30, 2021 05:25

Fix tests

98015bf

shwstppr reviewed Jul 30, 2021

View reviewed changes

Comment thread .../src/main/java/com/cloud/kubernetes/cluster/actionworkers/KubernetesClusterActionWorker.java Outdated

shwstppr reviewed Jul 30, 2021

View reviewed changes

Comment thread .../src/main/java/com/cloud/kubernetes/cluster/actionworkers/KubernetesClusterActionWorker.java

shwstppr reviewed Jul 30, 2021

View reviewed changes

Comment thread .../src/main/java/com/cloud/kubernetes/cluster/actionworkers/KubernetesClusterActionWorker.java Outdated

shwstppr reviewed Jul 30, 2021

View reviewed changes

Addressing comments

1db057b

davidjumani mentioned this pull request Jul 30, 2021

[DISCUSS] Work on next release apache/cloudstack-kubernetes-provider#16

Closed

Update plugins/integrations/kubernetes-service/src/main/java/com/clou…

6575a56

…d/kubernetes/cluster/actionworkers/KubernetesClusterActionWorker.java Co-authored-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

yadvr reviewed Aug 3, 2021

View reviewed changes

yadvr approved these changes Aug 3, 2021

View reviewed changes

GabrielBrascher reviewed Aug 3, 2021

View reviewed changes

nvazquez reviewed Aug 4, 2021

View reviewed changes

Comment thread ...e/src/main/java/com/cloud/kubernetes/cluster/actionworkers/KubernetesClusterStartWorker.java Outdated

nvazquez approved these changes Aug 4, 2021

View reviewed changes

Addressing comments

e4ed9c5

shwstppr closed this Aug 8, 2021

shwstppr reopened this Aug 8, 2021

yadvr approved these changes Aug 9, 2021

View reviewed changes

yadvr merged commit 3c8c704 into apache:main Aug 9, 2021

weizhouapache mentioned this pull request May 6, 2025

Cloudstack kubernetes Provider is not deployed in a shared network and routed mode isolated/vpc network #10795

Closed

Conversation

davidjumani commented Jul 29, 2021

Description

Types of changes

Feature/Enhancement Scale

How Has This Been Tested?

Uh oh!

davidjumani commented Jul 29, 2021

Uh oh!

blueorangutan commented Jul 29, 2021

Uh oh!

blueorangutan commented Jul 29, 2021

Uh oh!

blueorangutan commented Jul 30, 2021

Uh oh!

yadvr commented Jul 30, 2021

Uh oh!

davidjumani commented Jul 30, 2021

Uh oh!

davidjumani commented Jul 30, 2021

Uh oh!

blueorangutan commented Jul 30, 2021

Uh oh!

blueorangutan commented Jul 30, 2021

Uh oh!

Uh oh!

Uh oh!

Uh oh!

shwstppr left a comment

Choose a reason for hiding this comment

Uh oh!

davidjumani commented Jul 30, 2021

Uh oh!

blueorangutan commented Jul 30, 2021

Uh oh!

davidjumani commented Jul 30, 2021

Uh oh!

blueorangutan commented Jul 30, 2021

Uh oh!

blueorangutan commented Jul 30, 2021

Uh oh!

davidjumani commented Jul 31, 2021

Uh oh!

blueorangutan commented Jul 31, 2021

Uh oh!

davidjumani commented Aug 2, 2021

Uh oh!

blueorangutan commented Aug 2, 2021

Uh oh!

blueorangutan commented Aug 2, 2021

Uh oh!

davidjumani commented Aug 2, 2021

Uh oh!

blueorangutan commented Aug 2, 2021

Uh oh!

blueorangutan commented Aug 2, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yadvr left a comment

Choose a reason for hiding this comment

Uh oh!

nvazquez commented Aug 3, 2021

Uh oh!