Trying to figure out how to lock down my CI runners as much as possible.
-
@azonenberg would it be possible to mirror the github repo to a private forge and then only allow your runners to access that forge specifically?
@haskal that seems like a lot of work and complexity especially if i want to be testing PRs etc.
I think i'm better off putting a proxy between the runners and the outside world and filtering HTTP traffic there
-
@haskal that seems like a lot of work and complexity especially if i want to be testing PRs etc.
I think i'm better off putting a proxy between the runners and the outside world and filtering HTTP traffic there
@haskal I already have a private apt mirror so they don't have to phone home to install packages, they can only hit the internal apt-cacher-ng instance
-
@haskal I already have a private apt mirror so they don't have to phone home to install packages, they can only hit the internal apt-cacher-ng instance
@haskal (have not yet set up equivalent for non-debian distros though)
-
@whitequark i mean at a high level yes that seems the way to go. but at a more applied/practical level how does one create a "allow http to this one domain or even a specific range of URLs within it" http firewall? never actually done this
@azonenberg this is somewhat cursed but you can use nginx to do it. think of it as doing the inverse of the usual "reverse proxy" setup (so "reverse reverse proxy" I guess?): set up a server for "github.com" and location for "/user/repo" then proxy_pass it to actual github.com
obviously needs your cert to be trusted; and HPKP doesn't exist anymore so I don't think anything will complain that doesn't hardcode GitHub's TLS cert
-
@whitequark i mean at a high level yes that seems the way to go. but at a more applied/practical level how does one create a "allow http to this one domain or even a specific range of URLs within it" http firewall? never actually done this
@azonenberg @whitequark Typically if you're using HTTPS through a squid proxy or similar without some sort of MITM CA hacks you can only restrict by domain name - the hosts will CONNECT to the proxy (instead of GET, etc), the ACLs will apply to the hostname, then it's effectively plain TCP proxying at that point.
-
Trying to figure out how to lock down my CI runners as much as possible.
They need to be able to reach out to GitHub to HTTPS check out the source code, but I can't easily add a firewall rule for "can only clone this one repo from github".
Ideas beyond "open port 443 outbound to the entire internet"?
@azonenberg Github has an API to fetch their currently used IP ranges for various services which you could use to setup firewall rules (not sure how often they change in practice)
-
@haskal (have not yet set up equivalent for non-debian distros though)
@azonenberg is https clone required? could you use ssh with CI-specific "deploy keys"?
-
@azonenberg is https clone required? could you use ssh with CI-specific "deploy keys"?
@haskal then i have to do ssh filtering to avoid opening 22 outbound to the Internet which seems even harder?
-
@haskal then i have to do ssh filtering to avoid opening 22 outbound to the Internet which seems even harder?
@azonenberg only giving your CI a repo-specific deploy key would effectively enforce the "can only clone this one repo" rule
-
@azonenberg this is somewhat cursed but you can use nginx to do it. think of it as doing the inverse of the usual "reverse proxy" setup (so "reverse reverse proxy" I guess?): set up a server for "github.com" and location for "/user/repo" then proxy_pass it to actual github.com
obviously needs your cert to be trusted; and HPKP doesn't exist anymore so I don't think anything will complain that doesn't hardcode GitHub's TLS cert
@whitequark i have my own in-house TLS CA i can use in a pinch. Or a tls stripping proxy and run cleartext http client side (proxy and builder will live on same physical vm host so I'm not too worried about tampering)
-
@azonenberg this is somewhat cursed but you can use nginx to do it. think of it as doing the inverse of the usual "reverse proxy" setup (so "reverse reverse proxy" I guess?): set up a server for "github.com" and location for "/user/repo" then proxy_pass it to actual github.com
obviously needs your cert to be trusted; and HPKP doesn't exist anymore so I don't think anything will complain that doesn't hardcode GitHub's TLS cert
@whitequark @azonenberg this is called a "forward proxy" and used to be a lot more common
traditional use case was caching requests for a large enterprise, back when bandwidth was a lot more expensive. in the pre-HTTPS era this would even be set up to work transparently by intercepting connections on border routers
btw, there's a very obvious situation where this still gets used today: url blocking for restrictive environments such as banks or, much more commonly, the compulsory schooling environment
-
@azonenberg @whitequark Typically if you're using HTTPS through a squid proxy or similar without some sort of MITM CA hacks you can only restrict by domain name - the hosts will CONNECT to the proxy (instead of GET, etc), the ACLs will apply to the hostname, then it's effectively plain TCP proxying at that point.
@bencc @whitequark This is a closed system and I already have an in house TLS PKI for other reasons. I have no problem doing a MITM CA if that's the way to do it.
-
@whitequark @azonenberg this is called a "forward proxy" and used to be a lot more common
traditional use case was caching requests for a large enterprise, back when bandwidth was a lot more expensive. in the pre-HTTPS era this would even be set up to work transparently by intercepting connections on border routers
btw, there's a very obvious situation where this still gets used today: url blocking for restrictive environments such as banks or, much more commonly, the compulsory schooling environment
@r @azonenberg yes, I know that; what I mean is that virtually nobody runs nginx, specifically as a forward proxy, it would be normally Squid or something. so since nginx is almost exclusively used as a reverse proxy, it becomes a reverse reverse proxy
like a horseful horseless carriageI did use nginx in this configuration back in HK when I needed to make Conda work on a 10 Mbps (not a typo) fiber line
-
@r @azonenberg yes, I know that; what I mean is that virtually nobody runs nginx, specifically as a forward proxy, it would be normally Squid or something. so since nginx is almost exclusively used as a reverse proxy, it becomes a reverse reverse proxy
like a horseful horseless carriageI did use nginx in this configuration back in HK when I needed to make Conda work on a 10 Mbps (not a typo) fiber line
@whitequark @azonenberg oh hah, yes, brilliant
-
Trying to figure out how to lock down my CI runners as much as possible.
They need to be able to reach out to GitHub to HTTPS check out the source code, but I can't easily add a firewall rule for "can only clone this one repo from github".
Ideas beyond "open port 443 outbound to the entire internet"?
@azonenberg You might be able to
- find a document with a narrower set of address blocks that you can allowlist
- secure transport layer (tls/ssh) intercepting proxy (yuck) -
@whitequark @azonenberg oh hah, yes, brilliant
@r @whitequark anyway one of the challenges is that i do need to enable cloning some third party repos and I might have to figure out something for forks in PRs.
But allowing "all of github" is a good starting point and certainly better than letting the builders out to the internet at large
-
@bencc @whitequark This is a closed system and I already have an in house TLS PKI for other reasons. I have no problem doing a MITM CA if that's the way to do it.
@azonenberg @whitequark possibly that's more work to set up, but doesn't rely on the entire software stack obeying http_proxy/https_proxy env vars for success. I'd not expect 100% there, so possibly MITM would be more reliable. Plus you get find out anything bundles its own CA list...
-
@azonenberg @whitequark possibly that's more work to set up, but doesn't rely on the entire software stack obeying http_proxy/https_proxy env vars for success. I'd not expect 100% there, so possibly MITM would be more reliable. Plus you get find out anything bundles its own CA list...
@bencc @whitequark i only intend to do git clones and nothing else
-
Trying to figure out how to lock down my CI runners as much as possible.
They need to be able to reach out to GitHub to HTTPS check out the source code, but I can't easily add a firewall rule for "can only clone this one repo from github".
Ideas beyond "open port 443 outbound to the entire internet"?
@azonenberg Can you use ssh instead of https for the checkout? Seems it might be easier to lock down to only authorized connections.
-
@azonenberg Can you use ssh instead of https for the checkout? Seems it might be easier to lock down to only authorized connections.
@lluad I ideally want to limit to specific repositories so i think https mitm/stripping proxy is the only viable route here
-
R relay@relay.infosec.exchange shared this topic
