Hello readers! Here on Tech Insider we share exciting Palo Alto Networks deployment journeys. But we haven’t looked at what happens behind the scenes, i.e., using programming languages, such as Go, to improve security and robustness.
I am a developer in the First Customer team in IT where we implement our own products and build solutions to solve use cases (Microsegmentation, anyone?). After 3 years of using Go as the main language in my team, I’d like to share a few useful third-party libraries that focus on the security aspect and help reduce the number of bugs. I wish someone had told me about this at the onset, so I hope these recommendations are useful to you.
BTW if you’re not a developer but have made it this far but don’t have the stomach for a highly technical post, please do check other Tech Insider posts such as data loss prevention or CSPM.
Go!
When developers think about technologies, Stack Overflow survey is the most useful reference. Go is a relatively young language compared to the most commonly used languages such as Python, Java, Node.js, TypeScript and C/C++. Of course a language that’s common may not be the most desirable for developers. Go, however, is one of the top languages that developers love and want to learn. For me, it’s a natural choice due to its simplicity, platform-independence, good concurrency support and statically typed system.
golangci-lint
Static code analysis is important because we tend to make mistakes as humans. For other languages, there are a lot of tools in this area, e.g., SonarQube, Error-Prone, NullAway and OWASP Dependency-Check for Java, Flake8 for Python. For Go, there’s one suite that contains ~90 different linters: golangci-lint.
In particular, “gosec” is one of the linters that focuses on security. There is a good list of the rules including potential SQL injection, command execution audits, file/directory permission, missing random seed and more. Golangci-lint is flexible in enabling or disabling individual linters and also their configuration, e.g., allowlist a subset of rules.
Here’s the result of the gosec warnings in my recent project on IT asset database application. I will put a comment to ignore the G101 here since these are just placeholder credentials for the default values. However, it’s still a nice-to-have check. G402 is valid as I made a shortcut of disabling the TLS certification verification and I have to tighten this using the right CA pool. G601 can be an issue if the for loop alias value might change in an asynchronous way. If you are not sure what the rule is about, you may find more information in the gosec documentation or just simply search the rule title on the internet.
Preferably at the early stage of any Go project, integrating golangci-lint in the CI workflow or in various integrations can go a long way to avoid bugs and security issues.
Resty
In the First Customer team of Palo Alto Networks, the majority of our projects deal with HTTP APIs. I’m sure many of you have followed an example like this one to start hitting various API endpoints. However, we need to add many more lines of code to handle POST with a body, parse a JSON response to a struct, reuse an authentication scheme, or simply add a timeout to the http client. Surely, you can write all these in the native Go code. Do we have any library to help as we probably want to use python-requests rather than urllib in Python similarly?
Resty is the winner! It’s amazing just looking at the whole features list. These are my favorite:
- Chainable methods
- Automatic marshal and unmarshal for JSON content type
- Different client settings like Timeout, TLSClientConfig or Transport
- Request and Response middlewares
- Debug mode
Of these, #4 is very important to me because I can add a rate limiter to cap the request speed and deal with response error in a centralized place. Here’s a small example of setting up my DRY (don’t repeat yourself) HTTP client using the OnBeforeRequest and OnAfterResponse (gist):
My personal experience of using Resty yields ~3x less lines of code by removing all the copy and pastes, and error handling. With less code, there’s less chance of mistake and bug, hence, you get more security.
dockertest
In the software world, unit-testing is insufficient or an afterthought most of the time. There needs to be improvement, even in Palo Alto Networks IT. As you may know, unit-testing is important in software development because it can check that all the individual parts are working as intended, especially after adding new code or refactoring. However, unit testing database code in particular is hard to get an effective result. Normally, people will try to mock the database response using a mock library like go-sqlmock. The other option is to use an in-memory database (e.g., HSQLDB or Derby in Java) or SQLite (file database) to make the unit test run without setting up a real database. Of these two approaches, there’s still no guarantee your SQLs are still working perfectly in the real databases.
Docker to the rescue! One can start a real database from scratch, create all the tables and stored procedures using migrate, and destroy the database by removing the containers. Also multiple database instances can happen simultaneously as the container ports are different in multiple containers if you are running the tests in parallel. While we can use the official docker engine SDK to interact with the docker daemon, dockertest has a simpler and better interface for the testing purpose. Here’s my example (minus the error handling) of launching a PostgreSQL for my tests (gist):
After using dockertest, I am more confident my database schema and SQLs are working correctly in my targeted database. What’s more important is that they are more interesting to write.
And More
Beside the three really useful libraries I’ve talked about, there are a few smaller ones that may be of interest to you:
chi – HTTP router
zerolog – fast logger that understands types and supports structured logging
GoDS – various data structures in Go (treeset and treemap etc)
cast – dynamic data processing to different Go data type
cli by urfave – simple CLI builder that’s more lightweight than the other popular cobra
Go is already a very powerful and stable language natively with a vision. I hope my introduction of these third-party libraries can give you an even more enjoyable experience writing code, specifically, building robust Go programs with security checks and testing. Strong programming is the foundation to everything that you build in technology!
