8000 Bare metal k8s deployments by Shillaker · Pull Request #614 · faasm/faasm · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Bare metal k8s deployments #614

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Mar 14, 2022
Merged

Bare metal k8s deployments #614

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
87283ff
Instructions and task for setting up against bare metal k8s
Shillaker Mar 11, 2022
2b277d8
Latest cpp and knative bare metal config
Shillaker Mar 11, 2022
71b38cd
Revert update to cpp
Shillaker Mar 11, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions docs/source/kubernetes.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,8 @@ number of replicas.

The deploy can take up to a couple of minutes.

## Faasm config file

Once everything has started up, Faasm should also generate a config file,
`faasm.ini` at root of this project. This contains the relevant endpoints for
accessing the Faasm deployment.
Expand All @@ -65,6 +67,24 @@ If you need to regenerate this, you can run:
inv knative.ini-file
```

*If you are running on a bare metal cluster (i.e. not as part of a managed k8s
service like AKS), you'll need to run the following instead:*

```bash
inv knative.ini-file --publicip <ip of one of your hosts>
```

This is because Faasm normally assumes different public IPs for the upload and
invocation services, which are created as k8s `LoadBalancer`s in a cloud
provider. The command above instead uses the underlying `NodePort`s from those
`LoadBalancers`, by hitting one of the underlying hosts directly.

Unfortunately it's often not possible to query this public IP from within
`kubectl` itself, so you have to tell Faasm what it is explicitly.

Also make sure that all the ports mentioned in the Faasm config file are
accessible via your hosts' firewall rules.

## Uploading and invoking functions

Once you have configured your `faasm.ini` file, you can use the Faasm, CPP,
Expand Down
119 changes: 70 additions & 49 deletions faasmcli/faasmcli/tasks/knative.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -217,7 +217,7 @@ def install(ctx, reverse=False):


@task
def ini_file(ctx, local=False):
def ini_file(ctx, local=False, publicip=None):
"""
Set up the faasm config file for interacting with k8s
"""
Expand All @@ -236,7 +236,6 @@ def ini_file(ctx, local=False):
worker_names = list()
worker_ips = list()
else:
print("\n----- Extracting info from k8s -----\n")
knative_host = _capture_cmd_output(
[
"kn",
Expand All @@ -249,42 +248,74 @@ def ini_file(ctx, local=False):
)
knative_host = knative_host.strip()

# NOTE: we have to remove the http:// prefix from this url otherwise Istio
# We have to remove the http:// prefix from this url otherwise Istio
# won't recognise it
knative_host = knative_host.replace("http://", "")

invoke_ip = _capture_cmd_output(
[
"kubectl",
"-n istio-system",
"get",
"service",
"istio-ingressgateway",
"-o 'jsonpath={.status.loadBalancer.ingress[0].ip}'",
]
)
invoke_port = 80

upload_ip = _capture_cmd_output(
[
"kubectl",
"-n faasm",
"get",
"service",
"upload-lb",
"-o 'jsonpath={.status.loadBalancer.ingress[0].ip}'",
]
)
upload_port = _capture_cmd_output(
[
"kubectl",
"-n faasm",
"get",
"service",
"upload-lb",
"-o 'jsonpath={.spec.ports[0].port}'",
]
)
# If we're running on bare metal or VMs, we need to use the IP of the
# host provided for both invocations and upload, but instead of using a
# different load balancer for each, we must use the underlying node port
if publicip:
print("\n----- Setting up bare metal k8s config -----\n")

invoke_ip = publicip
upload_ip = publicip

invoke_port = _capture_cmd_output(
[
"kubectl",
"-n istio-system",
"get",
"service",
"istio-ingressgateway",
"-o 'jsonpath={.spec.ports[?(@.name==\"http2\")].nodePort}'",
]
)

upload_port = _capture_cmd_output(
[
"kubectl",
"-n faasm",
"get",
"service",
"upload-lb",
"-o 'jsonpath={.spec.ports[0].nodePort}'",
]
)
else:
print("\n----- Extracting info from k8s -----\n")
invoke_ip = _capture_cmd_output(
[
"kubectl",
"-n istio-system",
"get",
"service",
"istio-ingressgateway",
"-o 'jsonpath={.status.loadBalancer.ingress[0].ip}'",
]
)
invoke_port = 80

upload_ip = _capture_cmd_output(
[
"kubectl",
"-n faasm",
"get",
"service",
"upload-lb",
"-o 'jsonpath={.status.loadBalancer.ingress[0].ip}'",
]
)
upload_port = _capture_cmd_output(
[
"kubectl",
"-n faasm",
"get",
"service",
"upload-lb",
"-o 'jsonpath={.spec.ports[0].port}'",
]
)

worker_names, worker_ips = _get_faasm_worker_pods()

Expand All @@ -293,11 +324,14 @@ def ini_file(ctx, local=False):

with open(FAASM_CONFIG_FILE, "w") as fh:
fh.write("[Faasm]\n")

# This comment line can't be outside of the Faasm section
fh.write("# Auto-generated at {}\n".format(datetime.now()))

fh.write("invoke_host = {}\n".format(invoke_ip))
fh.write("invoke_port = {}\n".format(invoke_port))
fh.write("upload_host = {}\n".format(upload_ip))
fh.write("upload_port= {}\n".format(upload_port))
fh.write("upload_port = {}\n".format(upload_port))
fh.write("knative_host = {}\n".format(knative_host))
fh.write("worker_names = {}\n".format(",".join(worker_names)))
fh.write("worker_ips = {}\n".format(",".join(worker_ips)))
Expand All @@ -317,16 +351,3 @@ def ini_file(ctx, local=False):
shell=True,
check=True,
)

print("\n----- Examples -----\n")

print("Invoke:")
print("curl -H 'Host: {}' http://{}".format(knative_host, invoke_ip))

print("\nUpload:")
print(
"curl -X PUT http://{}:{}/f/<user>/<func> -T <wasm_file>".format(
upload_ip, upload_port
)
)
print("")
0