구독해서 새 게시물에 대한 알림을 받으세요.

Building the simplest Go static analysis tool

2016-04-27

2분 읽기
이 게시물은 English로도 이용할 수 있습니다.

Go native vendoring (a.k.a. GO15VENDOREXPERIMENT) allows you to freeze dependencies by putting them in a vendor folder in your project. The compiler will then look there before searching the GOPATH.

The only annoyance compared to using a per-project GOPATH, which is what we used to do, is that you might forget to vendor a package that you have in your GOPATH. The program will build for you, but it won't for anyone else. Back to the WFM times!

I decided I wanted something, a tool, to check that all my (non-stdlib) dependencies were vendored.

At first I thought of using go list, which Dave Cheney appropriately called a swiss army knife, but while it can show the entire recursive dependency tree (format .Deps), there's no way to know from the templating engine if a dependency is in the standard library.

We could just pass each output back into go list to check for .Standard, but I thought this would be a good occasion to build a very simple static analysis tool. Go's simplicity and libraries make it a very easy task, as you will see.

First, loading the program

We use golang.org/x/tools/go/loader to load the packages passed as arguments on the command line, including the test files based on a flag.

var conf loader.Config
for _, p := range flag.Args() {
    if *tests {
        conf.ImportWithTests(p)
    } else {
        conf.Import(p)
    }
}
prog, err := conf.Load()
if err != nil {
    log.Fatal(err)
}
for p := range prog.AllPackages {
    fmt.Println(p.Path())
}

With these few lines we already replicated go list -f {{ .Deps }}!

The only missing loading feature here is wildcard (./...) support. That code is in the go tool source and it's unexported. There's an issue about exposing it, but for now packages are just copy-pasting it. We'll use a packaged version of that code, github.com/kisielk/gotool:

for _, p := range gotool.ImportPaths(flag.Args()) {

Finally, since we are only interested in the dependency tree today we instruct the parser to only go as far as the imports statements and we ignore the resulting "not used" errors:

conf.ParserMode = parser.ImportsOnly
conf.AllowErrors = true
conf.TypeChecker.Error = func(error) {}

Then, the actual logic

We now have a loader.Program object, which holds references to various loader.PackageInfo objects, which in turn are a combination of package, AST and types information. All you need to perform any kind of complex analysis. Not that we are going to do that today :)

We'll just replicate the go list logic to recognize stdlib packages and remove the packages passed on the command line from the list:

initial := make(map[*loader.PackageInfo]bool)
for _, pi := range prog.InitialPackages() {
    initial[pi] = true
}

var packages []*loader.PackageInfo
for _, pi := range prog.AllPackages {
    if initial[pi] {
        continue
    }
    if len(pi.Files) == 0 {
        continue // virtual stdlib package
    }
    filename := prog.Fset.File(pi.Files[0].Pos()).Name()
    if !strings.HasPrefix(filename, build.Default.GOROOT) ||
        !isStandardImportPath(pi.Pkg.Path()) {
        packages = append(packages, pi)
    }
}

Then we just have to print a warning if any remaining package is not in a /vendor/ folder:

for _, pi := range packages {
    if strings.Index(pi.Pkg.Path(), "/vendor/") == -1 {
        fmt.Println("[!] dependency not vendored:", pi.Pkg.Path())
    }
}

Done! You can find the tool here: https://github.com/FiloSottile/vendorcheck

Further reading

This document maintained by Alan Donovan will tell you more than I'll ever know about the static analysis tooling.

Note that you might be tempted to use go/importer and types.Importer[From] instead of x/go/loader. Don't do that. That doesn't load the source but reads compiled .a files, which can be stale or missing. Static analysis tools that spit out "package not found" for existing packages or, worse, incorrect results because of this are a pet peeve of mine.

If you now feel the urge to write static analysis tools, know that the CloudFlare Go team is hiring in London, San Francisco and Singapore!

Cloudflare에서는 전체 기업 네트워크를 보호하고, 고객이 인터넷 규모의 애플리케이션을 효과적으로 구축하도록 지원하며, 웹 사이트와 인터넷 애플리케이션을 가속화하고, DDoS 공격을 막으며, 해커를 막고, Zero Trust로 향하는 고객의 여정을 지원합니다.

어떤 장치로든 1.1.1.1에 방문해 인터넷을 더 빠르고 안전하게 만들어 주는 Cloudflare의 무료 애플리케이션을 사용해 보세요.

더 나은 인터넷을 만들기 위한 Cloudflare의 사명을 자세히 알아보려면 여기에서 시작하세요. 새로운 커리어 경로를 찾고 있다면 채용 공고를 확인해 보세요.
ToolsGoProgramming

X에서 팔로우하기

Filippo Valsorda|@filosottile
Cloudflare|@cloudflare

관련 게시물