contrib: Interactive cmd-line tool for analyzing apitrace profiles

Apitrace is able to generate profile data with CPU and GPU timing info
from trace data. This go-based tool provides support for reading,
parsing and dissecting this profile data. It provides several Interactive
commands for generating different data views. It also provides commands\
for comparing two related profiles. See README for more info.

BUG=None
TEST=None

Change-Id: I8d05862886bdd0bcf91ccae40e6dcbc9b97f0a54
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/dev-util/+/1992190
Reviewed-by: David Riley <davidriley@chromium.org>
Commit-Queue: Georges Winkenbach <gwink@chromium.org>
Tested-by: Georges Winkenbach <gwink@chromium.org>
diff --git a/contrib/gfx/perf-tools/.gitignore b/contrib/gfx/perf-tools/.gitignore
new file mode 100644
index 0000000..4240c46
--- /dev/null
+++ b/contrib/gfx/perf-tools/.gitignore
@@ -0,0 +1,12 @@
+# Ignore bin folders.
+*/bin/*
+
+# Visual Studio Code
+/.vscode/
+/src/.vscode/
+
+# Emacs.
+*~
+
+# Vim
+*.sw[a-p]
diff --git a/contrib/gfx/perf-tools/profile-analysis/Makefile b/contrib/gfx/perf-tools/profile-analysis/Makefile
new file mode 100644
index 0000000..88fe4bd
--- /dev/null
+++ b/contrib/gfx/perf-tools/profile-analysis/Makefile
@@ -0,0 +1,32 @@
+# Copyright 2020 The Chromium OS Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+
+GOCMD=go
+GOBUILD=$(GOCMD) build
+GOCLEAN=$(GOCMD) clean
+GOGET = $(GOCMD) get
+GOFORMAT = $(GOCMD) fmt
+
+ANALYZE_SRCS=main.go
+ANALYZE_BIN=./bin/analyze
+REMOTE_PACKS= \
+	golang.org/x/image/colornames \
+	gonum.org/v1/plot \
+	gonum.org/v1/plot/plotter \
+	github.com/chzyer/readline
+
+all: build
+build: import
+import:
+	$(GOGET) $(REMOTE_PACKS)
+build:
+	$(GOBUILD) -o $(ANALYZE_BIN) $(ANALYZE_SRCS)
+
+clean:
+	$(GOCLEAN)
+	rm -f $(ANALYZE_BIN)
+
+format:
+	$(GOFORMAT) main.go
+	$(GOFORMAT) analyze/*.go
diff --git a/contrib/gfx/perf-tools/profile-analysis/README.md b/contrib/gfx/perf-tools/profile-analysis/README.md
new file mode 100644
index 0000000..00c7e5d
--- /dev/null
+++ b/contrib/gfx/perf-tools/profile-analysis/README.md
@@ -0,0 +1,110 @@
+# Analyze
+Analyze is a command-line tool for analyzing profiles produced by apitrace. It is
+used to analyze a single profile or pairs of profiles produced by running apitrace
+on the same trace on different platforms.
+
+## Producing profiles
+Profiles are generated by running the apitrace command `glretrace` on a trace file.
+Analyze can parse profile info with GPU and CPU timing information. To produce
+such files, you must run `glretrace` with options `--pcpu` and `--pgpu`. Most
+of the time, you'll also want to use option `--min-cpu-time=0` to ensure that
+the profile includes all calls. Without it, only calls taking 1 microsecond or
+more will be captured.
+
+Example:
+``` bash
+glretrace --pcpu --pgpu --min-cpu-time=0 traces_linux_10181_payday2_release.trace > profile_data.txt
+```
+
+Beware! When generating profiles by running traces in a Virgl environment, generating
+GPU timing can significantly skew the CPU timing. In such cases, it is better to capture
+CPU and GPU timings in separate profiles. (For the curious mind, this is because
+glretrace insert query operations around each call. Round-tripping this query
+ops through Virgl adds to the CPU time.)
+
+## Building Analyze
+1. Make sure the go tools are installed on your machine.
+2. Within directory `.../src/platform/dev/contrib/gfx/perf-tools/profile-analysis`
+run `make`.
+
+The binary for analyze is dropped in
+`.../src/platform/dev/contrib/gfx/perf-tools/profile-analysis/bin`
+
+## Running Analyze
+Once you have a profile and you've successfully build analyze, you're ready to
+analyze your first profile. Run analyze from within the bin directory as follows:
+
+``` text
+./bin/analyze <path to profile_data.txt>
+```
+
+Profiles can be very large and it might take a few minutes to load the profile.
+Once that is done, analyze will enter console mode:
+
+``` text
+Reading profile_data.txt....
+6222 frames read from profile_data.txt
+->
+```
+
+To get started, let's type a command to show the 5 most expensive frame by average
+CPU time:
+
+``` text
+-> show-frames n=5 s=bycpuavg
+Profile: trime_trace_virgl.txt
+  frame num   calls           GPU total            CPU total
+---------------------------------------------------------------------
+          3   64629             353.7 mS                3.6 S
+       1119   18811             331.3 mS             513.4 mS
+        912   35751             754.5 uS             348.5 mS
+       5986    9316               4.7 mS             301.8 mS
+       5861    7192               3.6 mS             243.9 mS
+->
+```
+
+A few more useful tidbits about the console:
+1. Type `help` to get basic help and `help <command>` to get more detailed
+help for that command.
+2. The console supports history. Use the up/down arrows to move back and forth
+between older commands.
+3. The console has limited support for tab completion.
+4. Type `quit` to exit analyze. (Command history is preserved across runs.)
+
+### Analyzing dual profiles
+Analyze is even more useful when processing two related profiles together.
+Two profiles are "related" if they were generated from the same trace, albeit
+on different platforms. To do so, run analyze with the two profiles as follows:
+
+``` text
+./bin/analyze <profile_data1.txt> <profile_data2.txt>
+```
+
+After loading the two profiles, analyze will check that they are compatible. (It
+does so by ensuring that matching calls in the two profiles call into the same
+GL function.)
+
+Here's a sample command that works on two profiles simultaneously:
+
+``` text
+-> show-frame-details 1646 gt=0 ct=500000
+Frame details for frames #1646 to 1646
+                                                   GPU       CPU     GPU 1  CPU 1         GPU       CPU      GPU 2   CPU 2
+ frame                      call name    call #    Prof 1    Prof 1    %      %           Prof 2    Prof 2     %       %
+----------------------------------------------------------------------------------------------------------------------------
+  1646                glCompileShader  10442713    0.0 nS    1.8 mS   0.0%   3.8%         0.0 nS    1.2 mS   0.0%   0.9%
+  1646                  glLinkProgram  10442733    0.0 nS    7.5 mS   0.0%  15.8%         0.0 nS   20.9 mS   0.0%  14.9%
+  1646            glDrawRangeElements  10442826   30.8 uS  733.6 uS   0.8%   1.5%        62.2 uS   38.1 uS   0.4%   0.0%
+  1646                glCompileShader  10442841    0.0 nS    1.7 mS   0.0%   3.6%         0.0 nS    1.1 mS   0.0%   0.8%
+  1646                glCompileShader  10442845    0.0 nS    4.2 mS   0.0%   8.9%         0.0 nS   24.6 uS   0.0%   0.0%
+  1646                  glLinkProgram  10442866    0.0 nS    7.4 mS   0.0%  15.5%         0.0 nS   25.8 mS   0.0%  18.4%
+  1646            glDrawRangeElements  10442953   29.4 uS  772.0 uS   0.8%   1.6%        64.6 uS   31.5 uS   0.4%   0.0%
+  1646                        glFlush  10443734    0.0 nS   21.2 mS   0.0%  44.6%         0.0 nS  125.2 uS   0.0%   0.1%
+1 frames out of 1 shown, or 100.0%
+```
+
+When analyzing two related profiles, it is even more important to use option
+`--min-cpu-time=0` when generating the profiles with `glretrace`. That is because,
+without it, `glretrace` will filter out different calls on each platform. You will
+likely end up with non-matching subsets of GL calls in each profile. They will
+still load in analyze successfully, but they will be harder to compare accurately.
diff --git a/contrib/gfx/perf-tools/profile-analysis/analyze/console.go b/contrib/gfx/perf-tools/profile-analysis/analyze/console.go
new file mode 100644
index 0000000..63f10ff
--- /dev/null
+++ b/contrib/gfx/perf-tools/profile-analysis/analyze/console.go
@@ -0,0 +1,616 @@
+// Copyright 2020 The Chromium OS Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+package analyze
+
+import (
+	"fmt"
+	"io"
+	"math"
+	"sort"
+	"strconv"
+	"strings"
+
+	"github.com/chzyer/readline"
+)
+
+type commandDispatch struct {
+	helpInfo     string                    // A short help string for the command.
+	dispatchFunc func(args []string) error // The dispatch function to call.
+	moreHelpFunc func(args []string)       // Optional function to show more help.
+}
+
+// Map command name to command-dispatch info.
+type commandDispatchTable = map[string]commandDispatch
+
+// All commands tend to take similar options, encapsulated in this struct.
+type cmdOptions struct {
+	prof           *ProfileData
+	filterRegex    string
+	numItemsToShow int
+	lessThanFunc   func(dsi DualStatistics, dsj DualStatistics) bool
+	sortByChoice   string
+	gpuThresholdNs int
+	cpuThresholdNs int
+}
+
+// Console encapsulates an interactive console to issues profile-analysis
+// commands.
+type Console struct {
+	profileData1  *ProfileData
+	profileData2  *ProfileData
+	cmdDispatch   commandDispatchTable
+	exitRequested bool
+}
+
+// StartInteractive starts the console interactive mode.
+func (console *Console) StartInteractive(prof1 *ProfileData, prof2 *ProfileData) {
+	rl, err := readline.NewEx(&readline.Config{
+		Prompt:       "-> ",
+		AutoComplete: createCompleter(),
+		HistoryFile:  "/tmp/_profile_analyzer_hist_.tmp",
+	})
+
+	if err != nil {
+		fmt.Println(err.Error())
+		return
+	}
+	defer rl.Close()
+
+	console.cmdDispatch = make(map[string]commandDispatch)
+	console.buildDispatchTable()
+	console.profileData1 = prof1
+	console.profileData2 = prof2
+
+	for !console.exitRequested {
+		cmd, err := rl.Readline()
+		if err != nil {
+			if err == io.EOF {
+				break // Ctrl-D --> exit
+			} else {
+				continue // Ctrl-C --> discard cmd
+			}
+		}
+
+		var tokens = strings.Fields(cmd)
+		if len(tokens) > 0 {
+			err := console.execCommand(tokens)
+			if err != nil {
+				fmt.Println(err.Error())
+			}
+		}
+	}
+}
+
+// Create the command completer, used for tab-completion.
+func createCompleter() *readline.PrefixCompleter {
+	return readline.NewPrefixCompleter(
+		readline.PcItem("help",
+			readline.PcItem("call-stats"),
+			readline.PcItem("swap-prof"),
+			readline.PcItem("show-prof"),
+			readline.PcItem("show-calls"),
+			readline.PcItem("show-frames"),
+			readline.PcItem("show-frame-details"),
+			readline.PcItem("compare-profiles"),
+			readline.PcItem("plot-calls"),
+			readline.PcItem("quit")),
+		readline.PcItem("quit"),
+		readline.PcItem("call-stats"),
+		readline.PcItem("swap-prof"),
+		readline.PcItem("show-prof"),
+		readline.PcItem("show-calls"),
+		readline.PcItem("show-frames"),
+		readline.PcItem("show-frame-details"),
+		readline.PcItem("compare-profiles"),
+		readline.PcItem("plot-calls"),
+	)
+}
+
+func (console *Console) buildDispatchTable() {
+	table := &console.cmdDispatch
+	(*table)["help"] = commandDispatch{
+		"Show help info.",
+		func(args []string) error { return console.printHelp(args) },
+		nil}
+	(*table)["quit"] = commandDispatch{
+		"Exit the console and go back to your regular life.",
+		func(args []string) error { return console.doQuit(args) },
+		nil}
+	(*table)["call-stats"] = commandDispatch{
+		"(call-stats [p1|p2] f=regex) Print call stats for the calls identified by a regex.",
+		func(args []string) error { return console.doCallStats(args) },
+		moreHelpForCallStats}
+	(*table)["swap-prof"] = commandDispatch{
+		"Swap profile1 and profile2, a no-op if there's only one profile.",
+		func(args []string) error { return console.doSwapProfiles(args) },
+		nil}
+	(*table)["show-prof"] = commandDispatch{
+		"(show-prof) Show basic information for the available profile(s).",
+		func(args []string) error { return console.doShowProfileInfo(args) },
+		nil}
+	(*table)["show-calls"] = commandDispatch{
+		"(show-calls [p1|p2]  [n=xx] [s=byGpuAvg|byCpuAvg]) Show information for xx\n" +
+			"        most expensive calls for the selected profile",
+		func(args []string) error { return console.doShowCalls(args) },
+		moreHelpForShowCalls}
+	(*table)["show-frames"] = commandDispatch{
+		"(show-frames [p1|p2]  [n=xx] [s=byGpuAvg|byCpuAvg]) Show information for xx\n" +
+			"        most expensive frames for the selected profile",
+		func(args []string) error { return console.doShowFrames(args) },
+		moreHelpForShowFrame}
+	(*table)["show-frame-details"] = commandDispatch{
+		"(show-frame-details N1[-N2]  [gt=xxx] [ct=xxx]) Show detailed call\n" +
+			"        information for frame N1 to N2.",
+		func(args []string) error { return console.doShowFrameDetail(args) },
+		moreHelpForShowFrameDetails}
+	(*table)["compare-profiles"] = commandDispatch{
+		"(compare-profiles [n=xx] [s=byGpuAvg|byCpuAvg]) Show timing comparison information for xx\n" +
+			"        most expensive calls for prof1 / prof2",
+		func(args []string) error { return console.doCompareProfiles(args) },
+		showMoreHelpForCompareProfile}
+	(*table)["plot-calls"] = commandDispatch{
+		"(plot-calls [p1|p2] f=regex) Plot call-name usage per frames.",
+		func(args []string) error { return console.doGraphCallUsage(args) },
+		nil}
+}
+
+func (console *Console) execCommand(args []string) error {
+	for cmdName, dispatch := range console.cmdDispatch {
+		if cmdName == args[0] {
+			return dispatch.dispatchFunc(args[1:])
+		}
+	}
+
+	if len(args) > 0 && len(strings.Trim(args[0], "\n\r ")) > 0 {
+		return fmt.Errorf("%s is not a recognized command", args[0])
+	}
+	return nil
+}
+
+func (console *Console) parseTargetProfile(args []string, options *cmdOptions) (err error) {
+	options.prof = console.profileData1
+	for _, arg := range args {
+		switch {
+		case arg == "p1", arg == "P1":
+			options.prof = console.profileData1
+		case arg == "p2", arg == "P2":
+			if options.prof = console.profileData2; options.prof == nil {
+				err = fmt.Errorf("profile 2 is not available: %s", arg)
+				return
+			}
+		default:
+		}
+	}
+
+	return nil
+}
+
+func (console *Console) parseCommandOptions(args []string) (options cmdOptions, err error) {
+	options = cmdOptions{
+		prof:           console.profileData1,
+		numItemsToShow: 0,
+		lessThanFunc:   sortByDecCPUAvg,
+		filterRegex:    "",
+		sortByChoice:   "BYCPUAVG",
+		gpuThresholdNs: 10000,
+		cpuThresholdNs: 10000,
+	}
+
+	if err = console.parseTargetProfile(args, &options); err != nil {
+		return
+	}
+
+	// If the target profile has no GPU timing data, set the default GPU threshold
+	// to 0. Likewise for CPU timing data.
+	if options.prof.sumGPUTimeNs == 0 {
+		options.gpuThresholdNs = 0
+	}
+	if options.prof.sumCPUTimeNs == 0 {
+		options.cpuThresholdNs = 0
+	}
+
+	for _, arg := range args {
+		switch {
+		case strings.HasPrefix(arg, "n="), strings.HasPrefix(arg, "N="):
+			if options.numItemsToShow, err = strconv.Atoi(arg[2:]); err != nil {
+				err = fmt.Errorf("invalid item-count option: %s", arg)
+				return
+			}
+		case strings.ToUpper(arg) == "S=BYGPUAVG":
+			options.lessThanFunc = sortByDecGPUAvg
+			options.sortByChoice = "BYGPUAVG"
+		case strings.ToUpper(arg) == "S=BYCPUAVG":
+			options.lessThanFunc = sortByDecCPUAvg
+			options.sortByChoice = "BYCPUAVG"
+		case strings.HasPrefix(arg, "f="), strings.HasPrefix(arg, "F="):
+			options.filterRegex = arg[2:]
+		case strings.HasPrefix(arg, "gt="), strings.HasPrefix(arg, "GT="):
+			if options.gpuThresholdNs, err = strconv.Atoi(arg[3:]); err != nil {
+				err = fmt.Errorf("invalid GPU threshold value: %s", arg)
+				return
+			}
+		case strings.HasPrefix(arg, "ct="), strings.HasPrefix(arg, "CT="):
+			if options.cpuThresholdNs, err = strconv.Atoi(arg[3:]); err != nil {
+				err = fmt.Errorf("invalid CPU threshold value: %s", arg)
+				return
+			}
+		case arg == "p1", arg == "P1", arg == "p2", arg == "P2":
+			continue // Parsed above.
+		default:
+			err = fmt.Errorf("invalid option: %s", arg)
+			return
+		}
+	}
+
+	return
+}
+
+func (console *Console) parseFrameRange(args []string) (frame1 int, frame2 int, err error) {
+	if args == nil {
+		err = fmt.Errorf("No frame number specified")
+		return
+	}
+
+	var arg = args[0]
+	var parts = strings.Split(arg, "-")
+	if frame1, err = strconv.Atoi(parts[0]); err != nil {
+		err = fmt.Errorf("invalid frame range: %s", arg)
+		return
+	}
+
+	frame2 = frame1
+	if len(parts) > 1 {
+		if frame2, err = strconv.Atoi(parts[1]); err != nil {
+			err = fmt.Errorf("invalid frame range: %s", arg)
+			return
+		}
+	}
+
+	if frame1 > frame2 {
+		frame2, frame1 = frame1, frame2
+	}
+
+	err = nil
+	return
+}
+
+func (console *Console) doQuit(args []string) error {
+	console.exitRequested = true
+	return nil
+}
+
+func (console *Console) doCallStats(args []string) error {
+	var err error
+	var options cmdOptions
+	if options, err = console.parseCommandOptions(args); err != nil {
+		return err
+	}
+
+	if options.filterRegex == "" {
+		options.filterRegex = ".*" // I.e. show stats for everything.
+	}
+
+	var gpuStats, cpuStats Statistics
+	if gpuStats, cpuStats, err = GatherStatisticsForCallNameRegex(
+		options.prof, options.filterRegex); err != nil {
+		return err
+	}
+
+	if options.prof.GetTotalGPUTimeNs() > 0 {
+		fmt.Printf("GPU timing statistics for %s:\n", options.prof.label)
+		printStats("  ", &gpuStats, options.prof.GetTotalGPUTimeNs())
+	} else {
+		fmt.Printf("GPU timing not available for %s:\n", options.prof.label)
+	}
+
+	if options.prof.GetTotalCPUTimeNs() > 0 {
+		fmt.Printf("CPU timing statistics for %s:\n", options.prof.label)
+		printStats("  ", &cpuStats, options.prof.GetTotalCPUTimeNs())
+	} else {
+		fmt.Printf("CPU timing not available for %s:\n", options.prof.label)
+	}
+	return nil
+}
+
+func (console *Console) doSwapProfiles(args []string) error {
+	if console.profileData2 != nil {
+		console.profileData1, console.profileData2 = console.profileData2, console.profileData1
+		fmt.Printf("Done: profile1 = %s, profile2 = %s", console.profileData1.label,
+			console.profileData2.label)
+	}
+	return nil
+}
+
+func (console *Console) doShowProfileInfo(args []string) error {
+	fmt.Printf("Profile info for p1=%s:\n", console.profileData1.label)
+	fmt.Printf("  Number of frames: %d\n", console.profileData1.GetFrameCount())
+	fmt.Printf("  Total number of calls: %d\n", console.profileData1.GetCallCount())
+
+	if console.profileData2 != nil {
+		fmt.Printf("Profile info for p2=%s:\n", console.profileData2.label)
+		fmt.Printf("  Number of frames: %d\n", console.profileData2.GetFrameCount())
+		fmt.Printf("  Total number of calls: %d\n", console.profileData2.GetCallCount())
+	}
+	return nil
+}
+
+func (console *Console) doShowCalls(args []string) error {
+	var err error
+	var options cmdOptions
+	if options, err = console.parseCommandOptions(args); err != nil {
+		return err
+	}
+
+	stats := GatherStatisticsForAllCallNames(options.prof)
+	sort.Slice(stats, func(i, j int) bool {
+		return options.lessThanFunc(stats[i], stats[j])
+	})
+
+	var gpuPerCallWeight = 100.0 / math.Max(options.prof.GetTotalGPUTimeNs(), 1e-6)
+	var cpuPerCallWeight = 100.0 / math.Max(options.prof.GetTotalCPUTimeNs(), 1e-6)
+
+	fmt.Printf("Profile: %s\n", options.prof.label)
+	fmt.Printf("%30s %7s %20s %20s %20s %20s\n", "call", "count", " GPU|CPU avg   ",
+		" GPU|CPU max   ", " GPU|CPU min   ", "GPU|CPU % total")
+	fmt.Printf("--------------------------------------------------------------" +
+		"-----------------------------------------------------------\n")
+	for n, s := range stats {
+		fmt.Printf("%30s %7d %9s |%9s %9s |%9s %9s |%9s %8.1f%% |%7.1f%%\n", s.callName,
+			s.gpuStat.numSamples,
+			timingToString(s.gpuStat.GetAverage()), timingToString(s.cpuStat.GetAverage()),
+			timingToString(s.gpuStat.GetMax()), timingToString(s.cpuStat.GetMax()),
+			timingToString(s.gpuStat.GetMin()), timingToString(s.cpuStat.GetMin()),
+			s.gpuStat.GetSum()*gpuPerCallWeight, s.cpuStat.GetSum()*cpuPerCallWeight)
+
+		if options.numItemsToShow > 0 && n == options.numItemsToShow-1 {
+			break
+		}
+	}
+
+	return nil
+}
+
+func (console *Console) doShowFrames(args []string) error {
+	var err error
+	var options cmdOptions
+	if options, err = console.parseCommandOptions(args); err != nil {
+		return err
+	}
+
+	timing := GatherTimingForAllFrames(options.prof)
+	sortFunc := func(i int, j int) bool {
+		return timing[i].cpuTimeNs > timing[j].cpuTimeNs
+	}
+	if options.sortByChoice == "BYGPUAVG" {
+		sortFunc = func(i int, j int) bool {
+			return timing[i].gpuTimeNs > timing[j].gpuTimeNs
+		}
+	}
+	sort.Slice(timing, sortFunc)
+
+	fmt.Printf("Profile: %s\n", options.prof.label)
+	fmt.Printf("%11s %7s %20s %20s\n", "frame num", "calls", "   GPU total ", "  CPU total ")
+	fmt.Printf("---------------------------------------------------------------------\n")
+	for n, t := range timing {
+		gpuTime := float64(t.gpuTimeNs)
+		cpuTime := float64(t.cpuTimeNs)
+		fmt.Printf("%11d %7d %20s %20s\n", t.frameNum, t.callCount,
+			timingToString(gpuTime), timingToString(cpuTime))
+
+		if options.numItemsToShow > 0 && n == options.numItemsToShow-1 {
+			break
+		}
+	}
+
+	return nil
+}
+
+func (console *Console) doShowFrameDetail(args []string) error {
+	if len(args) < 1 {
+		return fmt.Errorf("no frame number specified")
+	}
+
+	var firstFrame, lastFrame int
+	var err error
+	if firstFrame, lastFrame, err = console.parseFrameRange(args); err != nil {
+		return err
+	}
+
+	var options cmdOptions
+	if options, err = console.parseCommandOptions(args[1:]); err != nil {
+		return err
+	}
+
+	fmt.Printf("Frame details for frames #%d to %d\n", firstFrame, lastFrame)
+	fmt.Printf("%6s %30s %9s %9s %9s %6s %6s      %9s %9s %7s %7s\n",
+		"", "", "",
+		"GPU   ", "CPU   ", "GPU 1", "CPU 1",
+		"GPU   ", "CPU   ", "GPU 2", "CPU 2")
+	fmt.Printf("%6s %30s %9s %9s %9s %6s %6s      %9s %9s %7s %7s\n",
+		"frame", "call name", "call #",
+		"Prof 1", "Prof 1", "%  ", "%  ",
+		"Prof 2", "Prof 2", "%  ", "%  ")
+	fmt.Printf("----------------------------------------------------------------" +
+		"------------------------------------------------------------\n")
+
+	var frameCount = 0
+	for frameNum := firstFrame; frameNum <= lastFrame; frameNum++ {
+		var addToFrameCount = 1
+		var frameData1 []CallInfo = GatherCallDataForFrame(console.profileData1, frameNum)
+		var totGPUTime1, totCPUTime1 int = GatherFrameTiming(console.profileData1, frameNum)
+
+		var frameData2 []CallInfo
+		var totGPUTime2, totCPUTime2 int
+		if console.profileData2 != nil {
+			frameData2 = GatherCallDataForFrame(console.profileData2, frameNum)
+			totGPUTime2, totCPUTime2 = GatherFrameTiming(console.profileData2, frameNum)
+		}
+
+		var callCount = len(frameData1)
+		var gpuWeight1 = 1.0
+		if totGPUTime1 > 0 {
+			gpuWeight1 = 100.0 / float64(totGPUTime1)
+		}
+		var cpuWeight1 = 1.0
+		if totCPUTime1 > 0 {
+			cpuWeight1 = 100.0 / float64(totCPUTime1)
+		}
+		var gpuWeight2 = 1.0
+		if totGPUTime2 > 0 {
+			gpuWeight2 = 100.0 / float64(totGPUTime2)
+		}
+		var cpuWeight2 = 1.0
+		if totCPUTime2 > 0 {
+			cpuWeight2 = 100.0 / float64(totCPUTime2)
+		}
+
+		for i := 0; i < callCount; i++ {
+			c1 := frameData1[i]
+			if c1.gpuDurationNs >= options.gpuThresholdNs && c1.cpuDurationNs >= options.cpuThresholdNs {
+				frameCount += addToFrameCount
+				addToFrameCount = 0
+
+				gpu1Ns := float64(c1.gpuDurationNs)
+				cpu1Ns := float64(c1.cpuDurationNs)
+				fmt.Printf("%6d %30s %9d %9s %9s %5.1f%% %5.1f%%", frameNum, c1.callName,
+					c1.callNum, timingToString(gpu1Ns), timingToString(cpu1Ns),
+					gpuWeight1*gpu1Ns, cpuWeight1*cpu1Ns)
+
+				if frameData2 == nil {
+					fmt.Println("")
+					continue
+				}
+
+				if i >= len(frameData2) {
+					fmt.Println("        Frame not available in prof 2")
+					continue
+				}
+
+				c2 := frameData2[i]
+				if c2.callName != c1.callName {
+					fmt.Printf("      Call name mismatch: %s\n", c2.callName)
+					continue
+				}
+
+				gpu2Ns := float64(c2.gpuDurationNs)
+				cpu2Ns := float64(c2.cpuDurationNs)
+				fmt.Printf("      %9s %9s %5.1f%% %5.1f%%",
+					timingToString(gpu2Ns), timingToString(cpu2Ns),
+					gpuWeight2*gpu2Ns, cpuWeight2*cpu2Ns)
+
+				fmt.Println("")
+			}
+		}
+	}
+
+	var totFrames = lastFrame - firstFrame + 1
+	fmt.Printf("%d frames out of %d shown, or %.1f%%\n",
+		frameCount, totFrames, 100.0*float32(frameCount)/float32(totFrames))
+	return nil
+}
+
+func (console *Console) doCompareProfiles(args []string) error {
+	if console.profileData2 == nil {
+		return fmt.Errorf("this command requires two profiles")
+	}
+
+	var err error
+	var options cmdOptions
+	if options, err = console.parseCommandOptions(args); err != nil {
+		return err
+	}
+
+	var stats1 = GatherStatisticsForAllCallNames(console.profileData1)
+	sort.Slice(stats1, func(i, j int) bool {
+		return options.lessThanFunc(stats1[i], stats1[j])
+	})
+
+	var outStats, compareStats = GatherComparativeStats(
+		stats1, console.profileData2, options.numItemsToShow)
+
+	fmt.Printf("Compare statistics: %s / %s\n",
+		console.profileData1.label, console.profileData2.label)
+	fmt.Printf("%30s %7s %20s %20s %20s %20s\n", "", "", "Prof 1     ", "Prof 2     ",
+		"Ratio p1/p2  ", "Diff p1-p2  ")
+	fmt.Printf("%30s %7s %20s %20s %20s %20s\n", "call", "count", "GPU|CPU      ",
+		"GPU|CPU      ", "GPU|CPU      ", "GPU|CPU      ")
+	fmt.Printf("---------------------------------------------------------------" +
+		"-------------------------------------------------------------\n")
+	for n, stat2 := range outStats {
+		stat1 := stats1[n]
+		comp := compareStats[n]
+		fmt.Printf("%30s %7d %9s |%9s %9s |%9s %9s |%9s %9s |%9s\n",
+			stat2.callName, stat2.gpuStat.numSamples,
+			timingToString(stat1.gpuStat.GetAverage()), timingToString(stat1.cpuStat.GetAverage()),
+			timingToString(stat2.gpuStat.GetAverage()), timingToString(stat2.cpuStat.GetAverage()),
+			ratioToString(comp.gpuAvgRatio), ratioToString(comp.cpuAvgRatio),
+			timingToString(comp.gpuAvgDiff), timingToString(comp.cpuAvgDiff))
+
+		if options.numItemsToShow > 0 && n == options.numItemsToShow-1 {
+			break
+		}
+	}
+
+	return nil
+}
+
+func (console *Console) doGraphCallUsage(args []string) error {
+	var err error
+	var options cmdOptions
+	if options, err = console.parseCommandOptions(args); err != nil {
+		return err
+	}
+
+	if options.filterRegex == "" {
+		return fmt.Errorf("no call name specified")
+	}
+
+	return PlotCallNameUsagePerFrame(options.prof, options.filterRegex)
+}
+
+func printStats(prefix string, stats *Statistics, totalTimeNs float64) {
+	fmt.Printf("%sSample count:       %d\n", prefix, stats.numSamples)
+	fmt.Printf("%sAverage call time:  %s\n", prefix, timingToString(stats.GetAverage()))
+	fmt.Printf("%sMax call time:      %s\n", prefix, timingToString(stats.GetMax()))
+	fmt.Printf("%sMin call time:      %s\n", prefix, timingToString(stats.GetMin()))
+	fmt.Printf("%sStandard dev:       %s\n", prefix, timingToString(stats.GetStdDeviation()))
+	fmt.Printf("%sTotal time in call: %s\n", prefix, timingToString(stats.GetSum()))
+	percentInCall := 100.0 * stats.GetSum() / totalTimeNs
+	fmt.Printf("%s%% of time in call: %.1f\n", prefix, percentInCall)
+}
+
+// Convert an incoming timing value in nanoseconds to a display string of the
+// form x.y<unit>, where the unit is one of "nS", "uS", "mS" or "S" depending
+// on the range of the input value.
+func timingToString(timing float64) string {
+	var unit = "nS"
+	var t = timing
+	switch {
+	case math.Abs(timing) >= 1e9:
+		t = timing * 1e-9
+		unit = "S"
+	case math.Abs(timing) >= 1e6:
+		t = timing * 1e-6
+		unit = "mS"
+	case math.Abs(timing) >= 1000.0:
+		t = timing * 0.001
+		unit = "uS"
+	}
+
+	return fmt.Sprintf("%.1f %s", t, unit)
+}
+
+func ratioToString(ratio float64) string {
+	fmtStr := "%5.2f"
+	switch {
+	case ratio >= 1e50:
+		return "INF!"
+	case ratio >= 1000.0:
+		fmtStr = "%9.0f"
+	case ratio <= 1.0:
+		fmtStr = "%7.5f"
+	}
+	return fmt.Sprintf(fmtStr, ratio)
+}
diff --git a/contrib/gfx/perf-tools/profile-analysis/analyze/graph.go b/contrib/gfx/perf-tools/profile-analysis/analyze/graph.go
new file mode 100644
index 0000000..7960d17
--- /dev/null
+++ b/contrib/gfx/perf-tools/profile-analysis/analyze/graph.go
@@ -0,0 +1,71 @@
+// Copyright 2020 The Chromium OS Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+package analyze
+
+import (
+	"fmt"
+	"golang.org/x/image/colornames"
+	"gonum.org/v1/plot"
+	"gonum.org/v1/plot/plotter"
+	"gonum.org/v1/plot/vg"
+	"gonum.org/v1/plot/vg/draw"
+	"os/exec"
+)
+
+const plotFileName = "./analyzer_calls.png"
+
+// PlotCallNameUsagePerFrame generates a scatter-plot of the number of calls
+// to GL functions that match the given regex per frame and launches Chrome
+// to show the resulting graph.
+func PlotCallNameUsagePerFrame(prof *ProfileData, callNameRegex string) error {
+	var calls []int
+	var err error
+	if calls, err = GatherNumCallsPerFrame(prof, callNameRegex); err != nil {
+		return err
+	}
+
+	pts := make(plotter.XYs, len(calls))
+	for i, callCount := range calls {
+		if callCount > 0 {
+			pts[i].X = float64(i)
+			pts[i].Y = float64(callCount)
+		}
+	}
+
+	var p *plot.Plot
+	if p, err = plot.New(); err != nil {
+		return err
+	}
+
+	p.Title.Text = fmt.Sprintf("Calls to %s per frame", callNameRegex)
+	p.X.Label.Text = "frame num"
+	p.Y.Label.Text = "call count"
+	p.Add(plotter.NewGrid())
+
+	var s *plotter.Scatter
+	if s, err = plotter.NewScatter(pts); err != nil {
+		return err
+	}
+	s.GlyphStyle.Color = colornames.Blue
+	s.Shape = draw.PlusGlyph{}
+
+	p.Add(s)
+
+	// Save the plot to a PNG file.
+	if err = p.Save(9*vg.Inch, 4*vg.Inch, plotFileName); err != nil {
+		return err
+	}
+
+	// Show plot. If 'display' is available (imagemagick), we'll use that.
+	// Otherwise we fallback to Chrome.
+	var cmd *exec.Cmd
+	_, err = exec.LookPath("display")
+	if err != nil {
+		cmd = exec.Command("/usr/bin/google-chrome", plotFileName)
+	} else {
+		cmd = exec.Command("display", plotFileName)
+	}
+	return cmd.Start()
+}
diff --git a/contrib/gfx/perf-tools/profile-analysis/analyze/help.go b/contrib/gfx/perf-tools/profile-analysis/analyze/help.go
new file mode 100644
index 0000000..da013b1
--- /dev/null
+++ b/contrib/gfx/perf-tools/profile-analysis/analyze/help.go
@@ -0,0 +1,109 @@
+// Copyright 2020 The Chromium OS Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+package analyze
+
+import (
+	"fmt"
+)
+
+const beginBold = "\033[1m"
+const endBold = "\033[0m"
+
+const helpForProfileOption = "  [p1 | p2] is useful when more than one profile was given when launching\n" +
+	"            the analyzer tool. Use 'p1' to select the first profile or 'p2'\n" +
+	"            to select the second profile. Default is p1.\n"
+
+const helpForCallNameRegex = "  call-name-regex is a regular expression that specifies the call names to\n" +
+	"            gather statistics for.\n"
+
+const helpForSortOption = "  s=byGpuAvg|byCpuAvg sorts the ouput in either decreasing average GPU-time\n" +
+	"            or decreasing average CPU-time. The default is byCpuAvg. This option is not case sensitive.\n" +
+	"            I.e. s=bycpuavg is the same as s=byCpuAvg.\n"
+
+func moreHelpForCallStats(args []string) {
+	fmt.Println("\nHelp for call-stats command:\n" +
+		"Show basic information for all calls that match the given regular expression.\n" +
+		"Syntax: call-stats [p1 | p2] f=call-name-regex\n" +
+		helpForProfileOption +
+		helpForCallNameRegex +
+		"\nExamples: call-stats p1 f=glDraw  or  call-stats f=glDraw")
+}
+
+func moreHelpForShowCalls(args []string) {
+	fmt.Println("\nHelp for show-calls command:\n" +
+		"Print the xx most expensive calls in profile p1 or p2, sorted in decreasing order.\n" +
+		"Syntax: show-calls [p1 | p2] n=xx [s=byGpuAvg|byCpuAvg]\n" +
+		helpForProfileOption +
+		helpForSortOption +
+		"\nExamples: show-calls p2 n=30 s=byCpuAvg,  show-calls n=40\n")
+}
+
+func moreHelpForShowFrame(args []string) {
+	fmt.Println("\nHelp for show-frames command:\n" +
+		"Print the xx most expensive frames in profile p1 or p2, sorted in decreasing order.\n" +
+		"Syntax: show-frames [p1 | p2] n=xx [s=byGpuAvg|byCpuAvg]\n" +
+		helpForProfileOption +
+		helpForSortOption +
+		"\nExamples: show-frames p2 n=30 s=byCpuAvg,  show-frames n=20\n")
+}
+
+func moreHelpForShowFrameDetails(args []string) {
+	fmt.Println("\nHelp for show-frame-details command:\n" +
+		"Print individual call information for a sequence of frames. The information printed\n" +
+		"includes the call name, average GPU and CPU time spent in the call and the percentage\n" +
+		"of the frame time spent in that call for the GPU and CPU. If a second profile is\n" +
+		"available, the corresponding information for that profile is printed side-by-side.\n" +
+		"The calls are printed in the order in which they occur in the frame.\n" +
+		"Syntax: show-frame-details N1[-N2] gt=nnn ct=mmm\n" +
+		helpForProfileOption +
+		helpForSortOption +
+		"  gt=nnn  only show calls that spend nnn nanoseconds or more in the GPU.\n" +
+		"          Default is 100000 or 0 if no GPU timing data is available.\n" +
+		"  ct=mmm  only show calls that spend mmm nanoseconds or more in the CPU.\n" +
+		"          Default is 100000 or 0 if no CPU timing data is available.\n" +
+		"\nExample: show-frame-details 100-120 gt=100000 ct=500000\n")
+}
+
+func showMoreHelpForCompareProfile(args []string) {
+	fmt.Println("\nHelp for compare-profiles command:\n" +
+		"Show side-by-side timing comparison for the xx most expensive calls taken from profile p1.\n" +
+		"The comparison shows the average CPU and GPU time for each call in both profiles as well\n" +
+		"as the ratio (time-for-p1)/(time-for-p2) and difference (time-for-p1) - (time-for-p2) for both\n" +
+		"profiles for each of the calls.\n" +
+		"Syntax: compare-profiles n=xx [s=byGpuAvg|byCpuAvg]\n" +
+		helpForProfileOption +
+		helpForSortOption +
+		"\nExample: compare-profiles n=30 s=byCpuAvg\n")
+}
+
+func (console *Console) printHelp(args []string) error {
+	// If args is not empty, look for a command name for which to print more
+	// help info.
+	if len(args) > 0 {
+		for cmdName, dispatch := range console.cmdDispatch {
+			if cmdName == args[0] {
+				if dispatch.moreHelpFunc != nil {
+					dispatch.moreHelpFunc(args)
+				} else {
+					fmt.Printf("No additional available for %s\n", cmdName)
+				}
+				return nil
+			}
+		}
+	}
+
+	// Print general help.
+	fmt.Println(
+		"\nType commands of the form: '-> command [options]'\n" +
+			"Example: '-> show-calls p2 n=30 s=bycpuavg' shows the 30 most expensive calls in profile p2\n" +
+			" sorted by decreasing cpu average time.\n" +
+			"Type '-> help command' for more help on a specific command.\n" +
+			"\nAvailable commands:")
+	for cmdName, info := range console.cmdDispatch {
+		fmt.Println(beginBold + cmdName + endBold + ": " + info.helpInfo)
+	}
+
+	return nil
+}
diff --git a/contrib/gfx/perf-tools/profile-analysis/analyze/profile_analyzer.go b/contrib/gfx/perf-tools/profile-analysis/analyze/profile_analyzer.go
new file mode 100644
index 0000000..842a61c
--- /dev/null
+++ b/contrib/gfx/perf-tools/profile-analysis/analyze/profile_analyzer.go
@@ -0,0 +1,284 @@
+// Copyright 2020 The Chromium OS Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+package analyze
+
+import (
+	"fmt"
+	"math"
+	"math/rand"
+	"regexp"
+	"runtime"
+	"sync"
+	"sync/atomic"
+	"time"
+)
+
+// CheckProfileEquivalence returns whether two profiles are equivalent. Two
+// profiles are equivalent if whatever each call numbers they have in common
+// calls into the same api name (call name). E.g. if call num 101 in prof1
+// calls glClear then call 101 in prof2 must also call glClear.
+func CheckProfileEquivalence(prof1 *ProfileData, prof2 *ProfileData) bool {
+	// Iterate over the call in the smaller profile.
+	if prof1.GetCallCount() > prof2.GetCallCount() {
+		prof1, prof2 = prof2, prof1
+	}
+
+	// Verifying all calls would take too much time. Instead we use a random # to
+	// verify about 1/10 call.
+	s1 := rand.NewSource(time.Now().UnixNano())
+	r1 := rand.New(s1)
+
+	// Feed randomly selected call-info from profile1 into channel callsFromProf1.
+	type callInfo struct {
+		callNum  int
+		callName string
+	}
+	var callsFromProf1 = make(chan callInfo, 32)
+	go func() {
+		for _, callRecord := range prof1.allCalls {
+			if r1.Intn(100) >= 90 {
+				var callNum = callRecord.callNum
+				var callNumData = prof1.callsByCallNum[callNum]
+				callsFromProf1 <- callInfo{callNum, callNumData.callName}
+			}
+		}
+		close(callsFromProf1)
+	}()
+
+	// Compare calls selected above with the corresponding calls in profile2.
+	var numCallsMatch int32 = 0
+	var numCallsMismatch int32 = 0
+	for c := range callsFromProf1 {
+		go func(c callInfo) {
+			if prof2.VerifyCallNum(c.callNum, c.callName) {
+				atomic.AddInt32(&numCallsMatch, 1)
+			} else {
+				atomic.AddInt32(&numCallsMismatch, 1)
+			}
+		}(c)
+
+		// Show progress.
+		n := atomic.LoadInt32(&numCallsMatch)
+		if (n & 0xffff) == 0 {
+			fmt.Print(".")
+		}
+	}
+
+	// The two profiles need not share all the same call nums. But we do expect
+	// to verify at least some call nums to be equivalent and no call should
+	// mismatch.
+	fmt.Printf("\n%d api calls match and %d calls do not match.\n",
+		numCallsMatch, numCallsMismatch)
+	return numCallsMatch > 0 && numCallsMismatch == 0
+}
+
+// GatherStatisticsForCallNameRegex gathers statistics information for all the
+// call names in the profile that match the given regex.
+func GatherStatisticsForCallNameRegex(prof *ProfileData, callNameRegEx string) (
+	gpuStat Statistics, cpuStat Statistics, err error) {
+
+	// Traverse all calls in profile with a filter based on regex.
+	var re *regexp.Regexp
+	if re, err = regexp.Compile(callNameRegEx); err != nil {
+		return
+	}
+
+	c := make(chan int, 64)
+	filter := func(c CallInfo) bool {
+		return re.MatchString(c.callName)
+	}
+	go func() {
+		prof.TraverseAllCalls(filter, c)
+	}()
+
+	// Gather statistic for the calls that match regex.
+	for callIndex := range c {
+		call := prof.GetCallRecordByIndex(callIndex)
+		gpuStat.AddSample(float64(call.gpuDurationNs))
+		cpuStat.AddSample(float64(call.cpuDurationNs))
+	}
+
+	if gpuStat.GetNumSamples() == 0 {
+		err = fmt.Errorf("ERROR: no call matched call-name regex %s", callNameRegEx)
+	}
+	return
+}
+
+// GatherStatisticsForAllCallNames gathers statistics info for all the available
+// call names in the given profile.
+func GatherStatisticsForAllCallNames(prof *ProfileData) (stats []CallNameStatistics) {
+	var wait sync.WaitGroup
+
+	// Gather per-call-name call data through an async goroutine that feeds into
+	// callDataChan.
+	callDataChan := make(chan CallNameData, 64)
+	go func() { prof.TraverseByCallName(callDataChan) }()
+
+	// Gather the per-call-name statistics through another async goroutine, which
+	// get stats data from outResultChan and store into out-array stats.
+	outResultChan := make(chan CallNameStatistics, 64)
+	go func() {
+		for r := range outResultChan {
+			stats = append(stats, r)
+			wait.Done()
+		}
+	}()
+
+	// The limiter is used to limit the number of goroutines below.
+	numCoroutines := runtime.NumCPU()
+	limiter := make(chan int, numCoroutines)
+
+	// Launch a series of async goroutines to calculate the timing statistics for
+	// each call name. Each goroutine feeds the stats data to channel
+	// outResultChan.
+	for callData := range callDataChan {
+		limiter <- 1 // Blocks if there are already too many goroutines in flight.
+		wait.Add(1)
+
+		go func(name string, callIndices []int) {
+			gpuStat := Statistics{}
+			cpuStat := Statistics{}
+			for _, i := range callIndices {
+				data := prof.GetCallRecordByIndex(i)
+				gpuStat.AddSample(float64(data.gpuDurationNs))
+				cpuStat.AddSample(float64(data.cpuDurationNs))
+			}
+
+			outResultChan <- CallNameStatistics{name, gpuStat, cpuStat}
+			<-limiter
+		}(callData.callName, callData.callIndices)
+	}
+
+	wait.Wait() // Wait until all goroutines are done.
+	return
+}
+
+// GatherComparativeStats gathers comparative statistics for two profiles.
+// Comparative statistics consists of the ratio and difference between the
+// average GPU and CPU time by call name for up to <count> calls in the profiles.
+func GatherComparativeStats(
+	statsProf1 []CallNameStatistics, prof2 *ProfileData, count int) (
+	statsProf2 []CallNameStatistics, compareStats []CompareStats) {
+
+	for i, callStats := range statsProf1 {
+		var callName = callStats.callName
+		var gpuStat2 = Statistics{}
+		var cpuStat2 = Statistics{}
+		var comparison = CompareStats{}
+		if callIndices := prof2.GetCallIndicesForName(callName); callIndices != nil {
+			for _, callIndex := range callIndices {
+				data := prof2.GetCallRecordByIndex(callIndex)
+				gpuStat2.AddSample(float64(data.gpuDurationNs))
+				cpuStat2.AddSample(float64(data.cpuDurationNs))
+			}
+		}
+
+		comparison.gpuAvgDiff = callStats.gpuStat.GetAverage() - gpuStat2.GetAverage()
+		comparison.cpuAvgDiff = callStats.cpuStat.GetAverage() - cpuStat2.GetAverage()
+
+		// Avoid divide by zero, when there's no GPU stats.
+		if math.Abs(gpuStat2.GetAverage()) >= 1e-6 {
+			comparison.gpuAvgRatio = callStats.gpuStat.GetAverage() / gpuStat2.GetAverage()
+		} else {
+			// A very large number that the console knows to ignore.
+			comparison.gpuAvgRatio = 1e200
+		}
+		// Avoid divide by zero, when there's no CPU stats.
+		if math.Abs(cpuStat2.GetAverage()) >= 1e-6 {
+			comparison.cpuAvgRatio = callStats.cpuStat.GetAverage() / cpuStat2.GetAverage()
+		} else {
+			// A very large number that the console knows to ignore.
+			comparison.cpuAvgRatio = 1e200
+		}
+
+		statsProf2 = append(statsProf2, CallNameStatistics{callName, gpuStat2, cpuStat2})
+		compareStats = append(compareStats, comparison)
+
+		if count > 0 && i == count-1 {
+			break
+		}
+	}
+
+	return
+}
+
+// GatherNumCallsPerFrame calculates the number of times api function(s) that
+// match the given regex are called in each frame and returns the result as
+// an array indexed by frame num.
+func GatherNumCallsPerFrame(prof *ProfileData, callNameRegEx string) ([]int, error) {
+	// Traverse all calls in profile with a filter based on regex.
+	var re *regexp.Regexp
+	var err error
+	if re, err = regexp.Compile(callNameRegEx); err != nil {
+		return nil, err
+	}
+
+	c := make(chan int, 20)
+	filter := func(c CallInfo) bool {
+		return re.MatchString(c.callName)
+	}
+	go func() {
+		prof.TraverseAllCalls(filter, c)
+	}()
+
+	callCount := make([]int, prof.GetFrameCount())
+	for callIndex := range c {
+		callRecord := prof.GetCallRecordByIndex(callIndex)
+		callCount[callRecord.frameNum]++
+	}
+
+	return callCount, nil
+}
+
+// GatherCallDataForFrame gather the call information for all the calls in a
+// frame identified by its frame number and returns it as an array of CallInfo.
+func GatherCallDataForFrame(prof *ProfileData, frameNum int) []CallInfo {
+	r := prof.GetCallRangeForFrame(frameNum)
+	var start = r.firstIndex
+	var end = r.lastIndex
+	var numCalls = end - start + 1
+	callData := make([]CallInfo, numCalls)
+	for i := 0; i < numCalls; i++ {
+		callData[i] = prof.GetCallDataByIndex(i + start)
+	}
+
+	return callData
+}
+
+// GatherFrameTiming returns the total GPU and CPU time spent in the given frame.
+func GatherFrameTiming(
+	prof *ProfileData, frameNum int) (totalGPUTimeNs int, totalCPUTimeNs int) {
+	r := prof.GetCallRangeForFrame(frameNum)
+	for i := r.firstIndex; i <= r.lastIndex; i++ {
+		call := prof.GetCallRecordByIndex(i)
+		totalGPUTimeNs += call.gpuDurationNs
+		totalCPUTimeNs += call.cpuDurationNs
+	}
+	return
+}
+
+// GatherTimingForAllFrames returns the timing information for all the frames
+// in the profile. Timing info includes the total GPU and CPU time in each
+// frame.
+func GatherTimingForAllFrames(prof *ProfileData) (timing []FrameTiming) {
+	numCoroutines := runtime.NumCPU()
+	timing = make([]FrameTiming, prof.GetFrameCount())
+	limiter := make(chan int, numCoroutines)
+
+	for f := 0; f < prof.GetFrameCount(); f++ {
+		limiter <- 1
+		go func(f int) {
+			r := prof.GetCallRangeForFrame(f)
+			gpuNs, cpuNs := GatherFrameTiming(prof, f)
+			timing[f].frameNum = f
+			timing[f].gpuTimeNs += gpuNs
+			timing[f].cpuTimeNs += cpuNs
+			timing[f].callCount = r.lastIndex - r.firstIndex + 1
+			<-limiter
+		}(f)
+	}
+
+	return
+}
diff --git a/contrib/gfx/perf-tools/profile-analysis/analyze/profile_data.go b/contrib/gfx/perf-tools/profile-analysis/analyze/profile_data.go
new file mode 100644
index 0000000..7ad2f38
--- /dev/null
+++ b/contrib/gfx/perf-tools/profile-analysis/analyze/profile_data.go
@@ -0,0 +1,228 @@
+// Copyright 2020 The Chromium OS Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+package analyze
+
+import (
+	"fmt"
+)
+
+// CallRecord encapsulates the data associated with each api call.
+// Note: apitrace numbers each GL-api call sequentially. That number is called
+// the call num.
+type CallRecord struct {
+	callNum       int // The call number for this data.
+	frameNum      int // The number of the frame that invokes this call.
+	gpuDurationNs int // Gpu duration of this call in nanoseconds.
+	cpuDurationNs int // Cpu duration of this call in nanoseconds.
+}
+
+// CallRange tracks a range of calls by first and last indices.
+type CallRange struct {
+	firstIndex int
+	lastIndex  int // Inclusive
+}
+
+// CallNumData contains basic call info associated with each call num.
+type CallNumData struct {
+	programID int    // ID of program that uses this call.
+	callName  string // The api call name, e.g. glBegin.
+}
+
+// CallInfo all the info associated with a given call.
+type CallInfo struct {
+	CallNumData
+	CallRecord
+}
+
+// ProfileData implements interface ProfileDataConsumer. It is used to collect
+// the profile data gathered while reading a file and store that data in a
+// format that makes analyzing it easier.
+type ProfileData struct {
+	// A label for this profile data.
+	label string
+
+	// Current frame number while accumulating profile data.
+	frameNum int
+
+	// Array of all GL-api calls stored in this profile.
+	allCalls []CallRecord
+
+	// Keep track of basic call info for each call num.
+	callsByCallNum map[int](CallNumData)
+
+	// Maps call name, such as glDrawRangeElements, to a list of all the
+	// call-record indices that invoke that function.
+	callsByCallName map[string]([]int)
+
+	// Maps a frame number to a range of call indices that belong to that frame.
+	callsByFrameNum []CallRange
+
+	// Current accumulated CPU and GPU time in nanoseconds
+	sumGPUTimeNs float64
+	sumCPUTimeNs float64
+}
+
+// Filter function: takes calldata and return true if accepted by the filter.
+// These filter functions are used by some of the traverse methods provided by
+// ProfileData.
+type Filter func(callData CallInfo) bool
+
+// CallNameData encapsulates a call name and the list of all call indices
+// that invoke that call-name function.
+type CallNameData struct {
+	callName    string
+	callIndices []int
+}
+
+// StartNewProfile starts gathering a new profile. Any existing data in prof
+// is cleared.
+func (prof *ProfileData) StartNewProfile(label string) {
+	prof.label = label
+	prof.allCalls = make([]CallRecord, 0, 10000)
+	prof.callsByCallNum = map[int](CallNumData){}
+	prof.callsByCallName = map[string]([]int){}
+	prof.callsByFrameNum = make([]CallRange, 0, 2000)
+}
+
+// EndFrame ends the current frame. Any data added after this call will be
+// associated with the next sequential frame.
+func (prof *ProfileData) EndFrame() {
+	prof.frameNum++
+}
+
+// AddCallData adds profile info for the given call number to the profile data.
+func (prof *ProfileData) AddCallData(callNum int, gpuDurationNs int,
+	cpuDurationNs int, programID int, callName string) error {
+
+	// Usually, call numbers are unique and monotonically increasing. However,
+	// when a profile is generated with the last frame repeated several time
+	// (--loop option in glretrace), duplicate call numbers are present. The
+	// duplicate call numbers must always have the same call name and program ID.
+	if existingData, isDuplicate := prof.callsByCallNum[callNum]; isDuplicate {
+		if existingData.callName != callName {
+			return fmt.Errorf(
+				"ERROR: Duplicate call ID <%d> with mismatched call name %s v.s. %s",
+				callNum, callName, existingData.callName)
+		}
+		if existingData.programID != programID {
+			return fmt.Errorf(
+				"ERROR: Duplicate call ID <%d> with mismatched program ID %d v.s. %d",
+				callNum, programID, existingData.programID)
+		}
+	} else {
+		prof.callsByCallNum[callNum] = CallNumData{programID, callName}
+	}
+
+	// Add the call info to the list of all calls.
+	var callIndex = len(prof.allCalls)
+	prof.allCalls = append(prof.allCalls,
+		CallRecord{callNum, prof.frameNum, gpuDurationNs, cpuDurationNs})
+
+	// Add the index for this call record to this call name.
+	var callList = prof.callsByCallName[callName]
+	prof.callsByCallName[callName] = append(callList, callIndex)
+
+	if len(prof.callsByFrameNum) == prof.frameNum {
+		// This is a new frame; create a new call range for it.
+		prof.callsByFrameNum = append(prof.callsByFrameNum, CallRange{callIndex, callIndex})
+	} else {
+		// Expend the call range for the current frame to include this new call index.
+		prof.callsByFrameNum[prof.frameNum].lastIndex = callIndex
+	}
+
+	// Accumulate total CPU and GPU time in nanoseconds.
+	prof.sumCPUTimeNs += float64(cpuDurationNs)
+	prof.sumGPUTimeNs += float64(gpuDurationNs)
+
+	return nil
+}
+
+// GetFrameCount returns the number of frames in this profile.
+func (prof *ProfileData) GetFrameCount() int {
+	return prof.frameNum
+}
+
+// GetCallCount returns the total number of calls in this profile.
+func (prof *ProfileData) GetCallCount() int {
+	return len(prof.allCalls)
+}
+
+// GetTotalGPUTimeNs returns the accumulated time spent in the GPU in
+// nanoseconds.
+func (prof *ProfileData) GetTotalGPUTimeNs() float64 {
+	return prof.sumGPUTimeNs
+}
+
+// GetTotalCPUTimeNs returns the accumulated time spent in the CPU in
+// nanoseconds.
+func (prof *ProfileData) GetTotalCPUTimeNs() float64 {
+	return prof.sumCPUTimeNs
+}
+
+// GetCallDataByIndex returns the profile data for the call with the given
+// call index.
+func (prof *ProfileData) GetCallDataByIndex(index int) CallInfo {
+	var callNum = prof.allCalls[index].callNum
+	return CallInfo{prof.callsByCallNum[callNum], prof.allCalls[index]}
+}
+
+// GetCallRecordByIndex returns the call record for the given call index.
+func (prof *ProfileData) GetCallRecordByIndex(index int) CallRecord {
+	return prof.allCalls[index]
+}
+
+// GetCallIndicesForName returns a list of all the call indices that use the
+// given call name.
+func (prof *ProfileData) GetCallIndicesForName(name string) []int {
+	return prof.callsByCallName[name]
+}
+
+// GetCallRangeForFrame returns the range of call indices called in the
+// given frame.
+func (prof *ProfileData) GetCallRangeForFrame(frameNum int) CallRange {
+	if frameNum >= 0 && frameNum < len(prof.callsByFrameNum) {
+		return prof.callsByFrameNum[frameNum]
+	}
+	return CallRange{0, 0}
+}
+
+// VerifyCallNum returns whether call callNum invokes api with callName.
+// If callNum is not in this profile, then this function also returns true.
+func (prof *ProfileData) VerifyCallNum(callNum int, callName string) bool {
+	if call, ok := prof.callsByCallNum[callNum]; ok {
+		return callName == call.callName
+	}
+	return true
+}
+
+// TraverseAllCalls traverses all the available profile data sequentially and
+// feeds the call indices for the calls that are accepted by <filter> to channel c.
+func (prof *ProfileData) TraverseAllCalls(filter Filter, c chan int) {
+	for callIndex, callRecord := range prof.allCalls {
+		if filter(CallInfo{prof.callsByCallNum[callRecord.callNum], callRecord}) {
+			c <- callIndex
+		}
+	}
+	close(c)
+}
+
+// TraverseByCallName traverses all the available profile data and feeds each
+// call indices and associated call name to channel c.
+func (prof *ProfileData) TraverseByCallName(c chan CallNameData) {
+	for name, callIndices := range prof.callsByCallName {
+		c <- CallNameData{name, callIndices}
+	}
+	close(c)
+}
+
+// TraverseCallsForFrame feeds the call info to the given channel for all the
+// calls in frame <frameNum>.
+func (prof *ProfileData) TraverseCallsForFrame(c chan CallInfo, frameNum int) {
+	r := prof.GetCallRangeForFrame(frameNum)
+	for i := r.firstIndex; i <= r.lastIndex; i++ {
+		c <- prof.GetCallDataByIndex(i)
+	}
+	close(c)
+}
diff --git a/contrib/gfx/perf-tools/profile-analysis/analyze/profile_reader.go b/contrib/gfx/perf-tools/profile-analysis/analyze/profile_reader.go
new file mode 100644
index 0000000..1597f86
--- /dev/null
+++ b/contrib/gfx/perf-tools/profile-analysis/analyze/profile_reader.go
@@ -0,0 +1,185 @@
+// Copyright 2020 The Chromium OS Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+package analyze
+
+import (
+	"bufio"
+	"fmt"
+	"os"
+	"path/filepath"
+	"strconv"
+	"strings"
+)
+
+// ProfileDataConsumer defines the interface that takes profile data from
+// ProfileReader.
+type ProfileDataConsumer interface {
+	StartNewProfile(label string)
+	EndFrame()
+	AddCallData(callNum int, gpuDurationNs int, cpuDurationNs int,
+		programID int, callName string) error
+}
+
+// ProfileReader reads apitrace profile data from a file, parses it and feeds
+// the profile data to a ProfileDataConsumer.
+type ProfileReader struct {
+	// Full path name to the profile data file.
+	filename string
+
+	// Consumes the profile data produced by the reader.
+	consumer ProfileDataConsumer
+
+	// Column indices for the various profile data items in each line of data.
+	idxColumnCallID      int
+	idxColumnGPUDuration int
+	idxColumnCPUDuration int
+	idxColumnProgramID   int
+	idxColumnCallName    int
+	idxMax               int
+}
+
+// ReadProfile opens the profile data file, parses it line by line and feeds
+// the profile data to the consumer.
+func (reader *ProfileReader) ReadProfile(
+	filename string, consumer ProfileDataConsumer) (err error) {
+
+	var file *os.File
+	if file, err = os.Open(filename); err != nil {
+		fmt.Fprintf(os.Stderr, "ERROR: Unable to open file <%s>!\n", filename)
+		return
+	}
+	defer file.Close()
+
+	reader.filename = filepath.Base(file.Name())
+	reader.consumer = consumer
+
+	var bufSize = 64 * 1024
+	var buffer = make([]byte, bufSize)
+	var scanner = bufio.NewScanner(file)
+	scanner.Buffer(buffer, bufSize)
+
+	if err = reader.parseHeader(scanner); err != nil {
+		return
+	}
+
+	return reader.parseProfileData(scanner)
+}
+
+// Read the first, header line from the profile and extract the column indices
+// for the columns we are interested in.
+// Sample header from profile:
+// # call no gpu_start gpu_dura cpu_start cpu_dura vsize_start vsize_dura \
+//        rss_start rss_dura pixels program name
+func (reader *ProfileReader) parseHeader(scanner *bufio.Scanner) (err error) {
+	if !scanner.Scan() {
+		return fmt.Errorf("ERROR: No header in profile <%s>", reader.filename)
+	}
+
+	var columns = strings.Split(strings.TrimLeft(scanner.Text()[1:], " "), " ")
+	for idx, columnName := range columns {
+		switch columnName {
+		case "no":
+			reader.idxColumnCallID = idx
+		case "gpu_dura":
+			reader.idxColumnGPUDuration = idx
+		case "cpu_dura":
+			reader.idxColumnCPUDuration = idx
+		case "program":
+			reader.idxColumnProgramID = idx
+		case "name":
+			reader.idxColumnCallName = idx
+		case "call", "gpu_start", "cpu_start", "vsize_start", "vsize_dura",
+			"rss_start", "rss_dura", "pixels":
+			// Ignore those columns.
+			continue
+		default:
+			err = fmt.Errorf("Error: unexpected column name in profile: %s", columnName)
+			return
+		}
+		if idx > reader.idxMax {
+			reader.idxMax = idx
+		}
+	}
+
+	reader.consumer.StartNewProfile(reader.filename)
+	return nil
+}
+
+// Parse all the profile lines that follow the header and feed the profile
+// data to reader.consumer.
+func (reader *ProfileReader) parseProfileData(scanner *bufio.Scanner) (err error) {
+	// Asynchronously read lines from the input file and feed them to the lines
+	// channel.
+	var lines = make(chan string, 32)
+	go func() {
+		for scanner.Scan() {
+			lines <- scanner.Text()
+		}
+		close(lines)
+	}()
+
+	var frameOpen = false
+	for line := range lines {
+		switch {
+		case strings.HasPrefix(line, "Rendered"):
+			// Ignore last line in file.
+			continue
+		case strings.HasPrefix(line, "#"):
+			// Skip comment lines.
+			continue
+		case line == "frame_end":
+			reader.consumer.EndFrame()
+			frameOpen = false
+		case strings.HasPrefix(line, "call "):
+			err = reader.parseCallLine(line)
+			if err != nil {
+				fmt.Println(err)
+				return
+			}
+			frameOpen = true
+		default:
+			err = fmt.Errorf("Error: unrecognized line in profile: %s", line)
+			return
+		}
+	}
+
+	// Ensure the last frame is ended.
+	if frameOpen {
+		reader.consumer.EndFrame()
+	}
+
+	err = scanner.Err()
+	return
+}
+
+// Parse a single call line and feed the profile data to reader.consumer.
+// Sample call line from profile:
+// call 354 635667360 3680 594454016 59840 0 0 0 0 0 0 glClear
+func (reader *ProfileReader) parseCallLine(line string) (err error) {
+	var callID, gpuDuration, cpuDuration, programID int
+	var tokens = strings.Split(strings.TrimRight(line, "\r\n"), " ")
+
+	// Ensure that the number of tokens match or exceeds the column indices.
+	if len(tokens) <= reader.idxMax {
+		return fmt.Errorf("Error: not enough columns in call line: %s: ", line)
+	}
+
+	if callID, err = strconv.Atoi(tokens[reader.idxColumnCallID]); err != nil {
+		return err
+	}
+	if gpuDuration, err = strconv.Atoi(tokens[reader.idxColumnGPUDuration]); err != nil {
+		return err
+	}
+	if cpuDuration, err = strconv.Atoi(tokens[reader.idxColumnCPUDuration]); err != nil {
+		return err
+	}
+	if programID, err = strconv.Atoi(tokens[reader.idxColumnProgramID]); err != nil {
+		return err
+	}
+
+	err = reader.consumer.AddCallData(
+		callID, gpuDuration, cpuDuration, programID, tokens[reader.idxColumnCallName])
+	return
+}
diff --git a/contrib/gfx/perf-tools/profile-analysis/analyze/statistics.go b/contrib/gfx/perf-tools/profile-analysis/analyze/statistics.go
new file mode 100644
index 0000000..739a9f8
--- /dev/null
+++ b/contrib/gfx/perf-tools/profile-analysis/analyze/statistics.go
@@ -0,0 +1,134 @@
+// Copyright 2020 The Chromium OS Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+package analyze
+
+import (
+	"math"
+)
+
+// Statistics stores sum, simple average, min, max and standard-deviation info
+// and provides convenience methods to gather and calculate these values.
+type Statistics struct {
+	sum               float64
+	min               float64
+	max               float64
+	averageAtN        float64
+	averageAtNminus1  float64
+	varianceAtN       float64
+	varianceAtNminus1 float64
+	numSamples        int
+}
+
+// DualStatistics is a generic interface that embeds the dual notion of
+// GPU and CPU statistics and method to access them.
+type DualStatistics interface {
+	gpuStats() *Statistics
+	cpuStats() *Statistics
+}
+
+// CallNameStatistics has GPU and CPU statistics associated with a specific call
+// name. E.g. GPU and CPU timing statistics for function glDrawRangeElements.
+type CallNameStatistics struct {
+	callName string
+	gpuStat  Statistics
+	cpuStat  Statistics
+}
+
+// CompareStats encapsulates the ratio and difference of GPU and CPU averages.
+type CompareStats struct {
+	gpuAvgRatio float64
+	cpuAvgRatio float64
+	gpuAvgDiff  float64
+	cpuAvgDiff  float64
+}
+
+// FrameTiming encapsulates a frame number with the number of api calls that
+// take place within that frame and the total GPU and CPU time spent in those
+// calls.
+type FrameTiming struct {
+	frameNum  int
+	callCount int
+	gpuTimeNs int
+	cpuTimeNs int
+}
+
+// AddSample adds a sample value to the Statistics object.
+func (stat *Statistics) AddSample(s float64) {
+	stat.numSamples++
+
+	// Numerically stable running mean and variance, per Donald Knuth’s Art of
+	// Computer Programming, Vol 2.
+	if stat.numSamples == 1 {
+		stat.sum = s
+		stat.max = s
+		stat.min = s
+		stat.averageAtN = s
+		stat.varianceAtN = 0
+	} else {
+		stat.max = math.Max(stat.max, s)
+		stat.min = math.Min(stat.min, s)
+		stat.sum += s
+		stat.averageAtN = stat.averageAtNminus1 + (s-stat.averageAtNminus1)/float64(stat.numSamples)
+		stat.varianceAtN = stat.varianceAtNminus1 + (s-stat.averageAtNminus1)*(s-stat.averageAtN)
+	}
+
+	stat.averageAtNminus1 = stat.averageAtN
+	stat.varianceAtNminus1 = stat.varianceAtN
+}
+
+// GetNumSamples returns the number of samples accumulated so far.
+func (stat *Statistics) GetNumSamples() int {
+	return stat.numSamples
+}
+
+// GetSum returns the sum of all samples added so far.
+func (stat *Statistics) GetSum() float64 {
+	return stat.sum
+}
+
+// GetMax returns the maximum of all samples added so far.
+func (stat *Statistics) GetMax() float64 {
+	return stat.max
+}
+
+// GetMin returns the minimum of all samples added so far.
+func (stat *Statistics) GetMin() float64 {
+	return stat.min
+}
+
+// GetAverage returns the average of all samples added so far.
+func (stat *Statistics) GetAverage() float64 {
+	return stat.averageAtN
+}
+
+// GetStdDeviation returns the standard deviation of all samples added so far.
+func (stat *Statistics) GetStdDeviation() float64 {
+	if stat.numSamples <= 1 {
+		return 0
+	}
+	return math.Sqrt(stat.varianceAtN / float64(stat.numSamples-1))
+}
+
+// Implement interface DualStatistics on CallNameStatistics.
+func (ds CallNameStatistics) gpuStats() *Statistics {
+	return &ds.gpuStat
+}
+
+// Implement interface DualStatistics on CallNameStatistics.
+func (ds CallNameStatistics) cpuStats() *Statistics {
+	return &ds.cpuStat
+}
+
+// A sort-compare function for sorting DualStatistics samples by decreasing
+// average GPU time.
+func sortByDecGPUAvg(dsi DualStatistics, dsj DualStatistics) bool {
+	return dsi.gpuStats().averageAtN > dsj.gpuStats().averageAtN
+}
+
+// A sort-compare function for sorting DualStatistics samples by decreasing
+// average CPU time.
+func sortByDecCPUAvg(dsi DualStatistics, dsj DualStatistics) bool {
+	return dsi.cpuStats().averageAtN > dsj.cpuStats().averageAtN
+}
diff --git a/contrib/gfx/perf-tools/profile-analysis/main.go b/contrib/gfx/perf-tools/profile-analysis/main.go
new file mode 100644
index 0000000..d815a8c
--- /dev/null
+++ b/contrib/gfx/perf-tools/profile-analysis/main.go
@@ -0,0 +1,116 @@
+// Copyright 2020 The Chromium OS Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+package main
+
+import (
+	"./analyze"
+	"flag"
+	"fmt"
+	"os"
+)
+
+type profileData = analyze.ProfileData
+type profileReader = analyze.ProfileReader
+type console = analyze.Console
+
+func printUsage() {
+	fmt.Fprintf(os.Stderr, "\nUsage: %s [options] profile1 [profile2]\n", os.Args[0])
+	fmt.Fprintf(os.Stderr,
+		"where profile1 and optional profile2 are profile data files generated\n"+
+			"with 'glretrace' using the same input trace file.\n\n"+
+			"Available options:\n")
+	flag.PrintDefaults()
+	os.Exit(2)
+}
+
+func readProfile(filename string, readDone chan bool) (*profileData, error) {
+	var profData *profileData = nil
+	var err error = nil
+
+	if filename != "" {
+		var reader = new(profileReader)
+		fmt.Fprintf(os.Stdout, "Reading %s.... \n", filename)
+		profData = new(profileData)
+
+		if err = reader.ReadProfile(filename, profData); err == nil {
+			fmt.Printf("%d frames read from %s\n", profData.GetFrameCount(), filename)
+		}
+	}
+
+	readDone <- true
+	return profData, err
+}
+
+func readProfileOrExitOnFailure(filename string, readDone chan bool) *profileData {
+	var err error = nil
+	var profData *profileData = nil
+	if profData, err = readProfile(filename, readDone); err != nil {
+		fmt.Fprintf(os.Stderr, "%s\n", err.Error())
+		os.Exit(1)
+	}
+
+	return profData
+}
+
+func exitIfFileMissing(filepath string) {
+	if _, err := os.Stat(filepath); err != nil {
+		fmt.Fprintf(os.Stderr, "ERROR: Could not open <%s>\n", filepath)
+		os.Exit(1)
+	}
+}
+
+func main() {
+	// Parsing the cmd-line arguments.
+	var argShowHelp bool
+	flag.BoolVar(&argShowHelp, "help", false, "Show this help message")
+	flag.Usage = printUsage
+	flag.Parse()
+
+	if argShowHelp {
+		printUsage()
+	}
+
+	// Validate the input files. Profile 1 is needed, profile 2 is optional.
+	if len(flag.Args()) == 0 {
+		fmt.Fprintf(os.Stderr, "ERROR: You must profile at least one profile\n")
+		printUsage()
+	}
+	var argProfileFile1 = flag.Args()[0]
+	var argProfileFile2 string
+	if len(flag.Args()) > 1 {
+		argProfileFile2 = flag.Args()[1]
+		exitIfFileMissing(argProfileFile2)
+	}
+
+	exitIfFileMissing(argProfileFile1)
+
+	// Read and parse profiles.
+	var waitReadDone = make(chan bool, 2)
+	var profData1 *profileData = nil
+	var profData2 *profileData = nil
+	go func() {
+		profData1 = readProfileOrExitOnFailure(argProfileFile1, waitReadDone)
+	}()
+	go func() {
+		profData2 = readProfileOrExitOnFailure(argProfileFile2, waitReadDone)
+	}()
+
+	for i := 0; i < 2; i++ {
+		<-waitReadDone
+	}
+
+	// If we have two profiles, verify that they are equivalent.
+	if profData2 != nil {
+		fmt.Printf("Verifying profile compatibility....")
+		if !analyze.CheckProfileEquivalence(profData1, profData2) {
+			fmt.Fprintln(os.Stderr,
+				"ERROR: these two profiles do not appear to be from the same trace.")
+			os.Exit(1)
+		}
+	}
+
+	cons := new(console)
+	cons.StartInteractive(profData1, profData2)
+}