i#6495: Handle invariant errors in x86 QEMU syscall templates (#6718)

Handles various invariant errors seen in system call trace templates
collected on x86 QEMU.

Modifies syscall trace template file format to use the
TRACE_MARKER_TYPE_SYSCALL_TRACE_START and
TRACE_MARKER_TYPE_SYSCALL_TRACE_END markers to show start and end
respectively of each syscall trace template, instead of separating them
using a TRACE_MARKER_TYPE_SYSCALL marker. This makes it easier to write
invariant checks that also work for the syscall trace template file (in
addition to an actual trace file injected with trace templates). Also adds
cache line size and page size markers to the template, similar to the
context switch sequence template file.

Handles cases where there are a different number of read/write records
than expected by the decoder; after iret, variants of xrstor, variants
of xsaves, and prefetch instrs.

Relaxes the PC discontinuity check after hlt, and within two instrs of
sti (which enables interrupts, so there may be an interrupt shortly
after, as empirically seen in some QEMU syscall trace templates).

Makes other misc changes to make sure the syscall trace template file
passes the invariant checker: add thread exit (since we already have a
thread start), relaxation of various invariant checks.

Adds and implements the instr_is_xrstor API that identifies variants of
the xrstor opcode, and adds supervisor versions of xsave to
instr_is_xsave.

Adds unit tests for these new scenarios. Added a TODO to handle other
arch equivalent versions of these scenarios.

Adds a new flag `-abort_on_invariant_error` which is true by default, to
allow the user to instruct the invariant checker to continue past
invariant errors (using `-no_abort_on_invariant_error`). This is helpful
since there are still a few instances of some invariant errors in the
syscall trace template that are harder to generalize and fix/ignore.

Issue: #6495
13 files changed
tree: 1e436797712234f0906560ad2278ee8d81e798d8
  1. .github/
  2. api/
  3. clients/
  4. core/
  5. ext/
  6. libutil/
  7. make/
  8. suite/
  9. third_party/
  10. tools/
  11. .clang-format
  12. .gitignore
  13. .gitmodules
  14. ACKNOWLEDGEMENTS
  15. CMakeLists.txt
  16. CONTRIBUTING.md
  17. CTestConfig.cmake
  18. License.txt
  19. README
  20. README.md
README.md

DynamoRIO

DynamoRIO logo

About DynamoRIO

DynamoRIO is a runtime code manipulation system that supports code transformations on any part of a program, while it executes. DynamoRIO exports an interface for building dynamic tools for a wide variety of uses: program analysis and understanding, profiling, instrumentation, optimization, translation, etc. Unlike many dynamic tool systems, DynamoRIO is not limited to insertion of callouts/trampolines and allows arbitrary modifications to application instructions via a powerful IA-32/AMD64/ARM/AArch64 instruction manipulation library. DynamoRIO provides efficient, transparent, and comprehensive manipulation of unmodified applications running on stock operating systems (Windows, Linux, or Android) and commodity IA-32, AMD64, ARM, and AArch64 hardware. Mac OSX support is in progress.

Existing DynamoRIO-based tools

DynamoRIO is the basis for some well-known external tools:

Tools built on DynamoRIO and available in the release package include:

  • The memory debugging tool Dr. Memory
  • The tracing and analysis framework drmemtrace with multiple tools that operate on both online (with multi-process support) and offline instruction and memory address traces:
  • The legacy processor emulator drcpusim
  • The “strace for Windows” tool drstrace
  • The code coverage tool drcov
  • The library tracing tool drltrace
  • The memory address tracing tool memtrace (drmemtrace's offline traces are faster with more surrounding infrastructure, but this is a simpler starting point for customized memory address tracing)
  • The memory value tracing tool memval
  • The instruction tracing tool instrace (drmemtrace's offline traces are faster with more surrounding infrastructure, but this is a simpler starting point for customized instruction tracing)
  • The basic block tracing tool bbbuf
  • The instruction counting tool inscount
  • The dynamic fuzz testing tool Dr. Fuzz
  • The disassembly tool drdisas
  • And more, including opcode counts, branch instrumentation, etc.: see API samples

Building your own custom tools

DynamoRIO‘s powerful API abstracts away the details of the underlying infrastructure and allows the tool builder to concentrate on analyzing or modifying the application’s runtime code stream. API documentation is included in the release package and can also be browsed online. Slides from our past tutorials are also available.

Downloading DynamoRIO

DynamoRIO is available free of charge as a binary package for both Windows and Linux. DynamoRIO's source code is available primarily under a BSD license.

Obtaining Help

Use the discussion list to ask questions.

To report a bug, use the issue tracker.

See also the DynamoRIO home page: http://dynamorio.org/