Creating an Interactive Security Sandbox

Cordon

#

I can finally start to implement cordon. My userspace helper simply takes in some information and shows a bubbletea dialog styled with lipgloss, and allows the user to approve or deny the operation.

With it, I can sandbox bash for example. This is useful if you ever want to curl ... | bash to watch what it is doing. It looks like this:

What is implemented right now is opening of files, and making a network connection. There are lots of holes in it. For example, you can still delete files anywhere (security_path_unlink is unimplemented). It also does not prevent tampering with the sandbox from within it, by editing eBPF maps or by trying to read xattrs from the fuse filesystem, or by killing the parent sandbox process (though this last one will also make you exit). However, at this time it is a decent proof of concept of the technique. The rest of it can be ironed out later.

Environment Variables

#

While this technique is great, there are a few fundamental restrictions which don’t make it as nice as Deno’s sandbox. One issue is environment variables. Deno.env.get (and the node compat process.env) trigger interactive prompts to the user which provide the name of the environment variable as context to the user so they can make the decision on if it is safe. This is simply not the way environment variables are handled at the OS level.

The execve syscall is used on Linux to replace the current process with a different executable:

int execve(const char *pathname, char *const _Nullable argv[], char *const _Nullable envp[]);

All environment variables are passed directly into the new binary at startup. All arguments and environment variables are placed directly on the stack by the kernel.

Then whatever language you use reads all these values before your main is called, and transforms them into a language specific API. The two things which are done are typically:

1. Convert the arguments and environment variables from C strings to the language string format. Lots of languages like Go and Rust have string types with a usize length value and a pointer to the data, so they need to initialize those and possibly copy the contents somewhere else.
2. Build a hashmap or similar out of the environment variables. libc exposes char *getenv(const char *name), and Rust exposes std::env::var(...) for example.

You might be able to see the problem here. There is no way to hide environment variables until they are read. Instead, they are all read on program startup, and shuffled around all over the place. There is no getenv syscall which we could interpose on.

Also, side note/fun fact. Setting an environment variable is fake. Any standard library providing a setenv like function, is just modifying the hashmap representation of the language. The kernel is never notified of such an edit, additional environment variable, or the deletion of one, even though it populated them in the first place. Additionally, languages like Go protect the map with a mutex. But with cgo enabled, os.Setenv instead uses libc’s setenv which is very thread unsafe, leading to a segfault in Go too in multithreaded programs. Indeed, cgo is not Go.

So, there is no reliable way to do language-independent environment variable interactive permissions.

One could potentially mask environment variable values with some garbage data of the same length before calling execve. This is so it still uses the same amount of memory as if they were there. Then, an eBPF uprobe could be inserted on language specific functions like getenv in libc, and edit the process’s memory to “unmask” the true value of the environment variable if it was approved. However, this approach is very brittle and it would be very tricky to implement. It also leaks the metadata of environment variable names as well as the lengths of their values. Though I don’t think it is that big of a deal a program knows that AWS_SECRET_ACCESS_KEY is set, if it can’t know its actual value.

Another approach could be to look at it heuristically. That is, start by allowing benign environment variables like HOME and PATH for example. Then, run strings or similar on the binary to see if any SCREAMING_SNAKE_CASE strings are statically defined. Then we can compare those potential matches to the currently set environment variables, and ask the user before the program starts if it wants to forward those on. This obviously won’t cover all programs, but it is probably better than nothing.

Network

#

You typically establish an internet connection with the connect syscall. However it is only possible to extract the IP which you want to connect to, not the domain. An IP address is pretty meaningless to a user trying to determine if something is secure or not. We also can’t look at the data sent because it is probably encrypted. And, the program might use DNS over TLS so we can’t even introspect based on DNS traffic (which is traditionally UDP).

What cordon does is it asks systemd-resolved for recently resolved requests to tries to reverse resolve the IP it got from connect to a recent query. This works when the program simply uses getadderinfo, but for a truly malicious program it will not resolve.

Actually, the worst case scenario is that it falsely reverse resolved, meaning the user may approve a domain which it is not actually being sent to. This can happen because the same IP can serve many different websites. We can thank IPv4 for not having nearly enough address space for this. For example, CDNs do this, and so does Cloudflare. An attacker can put their website behind Cloudflare, then find a benign website also behind Cloudflare which happens to share the same IP as the attacker. (This is an oversimplification because of anycast, but bear with me here). Then the attacker does a query to the benign site, which you approve. Then it sends an HTTP request to the attacker site with a different domain in the Host header, but because it has the same IP it looks to cordon and you like it simply is sending to the benign site again, so it will be approved.

There is basically no way to prevent this. This is why I have such a strong emphasis on restricting filesystem access. A malicious program cannot exfiltrate something it does not have access to. In my opinion, a program should either have filesystem read access, or network connect access, but not both. Though strict adherence to this rule would obviously imply whole classes of programs shouldn’t exist.

It would be possible to implement an L7 proxy and transparently route traffic through it from the sandboxed program using eBPF. That way, cordon could analyze HTTP traffic for example, to get more context to provide to the user. It could even look at the SNI of TLS connections to see what domain it wants to connect to. Though, in TLS 1.3 the SNI can be encrypted with Encrypted Client Hello (ECH), so it wouldn’t be able to grab it out in that case either. Also, the SNI can differ from the Host in the HTTP header anyways, so it’d be back to square one in terms of trying to understand where the traffic is actually heading.

Conclusion

#

Overall, cordon is not ready, and is still highly insecure. It turns out that it takes a lot of effort to build robust and secure software. I hope to spend more time in the future developing the project and seeing how far I can take it.

My end goal would be to have some kind of daemon watching for programs executing in directories with .git in it. Then it can automatically insert itself and approve per-project based permissions. The daemon could also run Trufflehog (or similar) on all files which your user processes attempt to read before letting the process read them, to help reduce the number of files the user has to approve.

If you’re interested in it, check out the source code at mbund/cordon for the full implementation, and for updates on the future status of the project.

In any case, here are the main takeaways I hope you have after reading this post:

• Sleepable eBPF programs can run any arbitrary userspace function with FUSE xattr RPC. This has potential for writing better EDR software for Linux.
• Ideally, interpreted runtimes should have interactive sandboxes like Deno. They can provide the most context to the user (for example, environment variables, which are not possible with a “native” solution)
• Use Landlock if you can, because the developer has the most context on what should or should not be accessed.
• Try to reduce the amount of third party code you depend on.