llvmbot wrote:
<!--LLVM PR SUMMARY COMMENT--> @llvm/pr-subscribers-lldb Author: Jason Molenda (jasonmolenda) <details> <summary>Changes</summary> The "process metadata" LC_NOTE allows for thread IDs to be specified in a Mach-O corefile. This extends the JSON recognzied in that LC_NOTE to allow for additional registers to be supplied on a per-thread basis. The registers included in a Mach-O corefile LC_THREAD load command can only be one of the register flavors that the kernel (xnu) defines in <mach/arm/thread_status.h> for arm64 -- the general purpose registers, floating point registers, exception registers. JTAG style corefile producers may have access to many additional registers beyond these that EL0 programs typically use, for instance TCR_EL1 on AArch64, and people developing low level code need access to these registers. This patch defines a format for including these registers for any thread. The JSON in "process metadata" is a dictionary that must have a `threads` key. The value is an array of entries, one per LC_THREAD in the Mach-O corefile. The number of entries must match the LC_THREADs so they can be correctly associated. Each thread's dictionary must have two keys, `sets`, and `registers`. `sets` is an array of register set names. If a register set name matches one from the LC_THREAD core registers, any registers that are defined will be added to that register set. e.g. metadata can add a register to the "General Purpose Registers" set that lldb shows users. `registers` is an array of dictionaries, one per register. Each register must have the keys `name`, `value`, `bitsize`, and `set`. It may provide additional keys like `alt-name`, that `DynamicRegisterInfo::SetRegisterInfo` recognizes. This `sets` + `registers` formatting is the same that is used by the `target.process.python-os-plugin-path` script interface uses, both are parsed by `DynamicRegisterInfo`. The one addition is that in this LC_NOTE metadata, each register must also have a `value` field, with the value provided in big-endian base 10, as usual with JSON. In RegisterContextUnifiedCore, I combine the register sets & registers from the LC_THREAD for a specific thread, and the metadata sets & registers for that thread from the LC_NOTE. Even if no LC_NOTE is present, this class ingests the LC_THREAD register contexts and reformats it to its internal stores before returning itself as the RegisterContex, instead of shortcutting and returning the core's native RegisterContext. I could have gone either way with that, but in the end I decided if the code is correct, we should live on it always. I added a test where we process save-core to create a userland corefile, then use a utility "add-lcnote" to strip the existing "process metadata" LC_NOTE that lldb put in it, and adds a new one from a JSON string. rdar://74358787 --- Patch is 41.58 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/144627.diff 11 Files Affected: - (modified) lldb/include/lldb/Symbol/ObjectFile.h (+15-2) - (modified) lldb/source/Plugins/ObjectFile/Mach-O/ObjectFileMachO.cpp (+41-20) - (modified) lldb/source/Plugins/ObjectFile/Mach-O/ObjectFileMachO.h (+2) - (modified) lldb/source/Plugins/Process/mach-core/CMakeLists.txt (+1) - (added) lldb/source/Plugins/Process/mach-core/RegisterContextUnifiedCore.cpp (+293) - (added) lldb/source/Plugins/Process/mach-core/RegisterContextUnifiedCore.h (+57) - (modified) lldb/source/Plugins/Process/mach-core/ThreadMachCore.cpp (+42-13) - (added) lldb/test/API/macosx/lc-note/additional-registers/Makefile (+11) - (added) lldb/test/API/macosx/lc-note/additional-registers/TestMetadataRegisters.py (+100) - (added) lldb/test/API/macosx/lc-note/additional-registers/add-lcnote.cpp (+384) - (added) lldb/test/API/macosx/lc-note/additional-registers/main.c (+11) ``````````diff diff --git a/lldb/include/lldb/Symbol/ObjectFile.h b/lldb/include/lldb/Symbol/ObjectFile.h index 43567592dd447..1b9ae1fb31a69 100644 --- a/lldb/include/lldb/Symbol/ObjectFile.h +++ b/lldb/include/lldb/Symbol/ObjectFile.h @@ -18,6 +18,7 @@ #include "lldb/Utility/Endian.h" #include "lldb/Utility/FileSpec.h" #include "lldb/Utility/FileSpecList.h" +#include "lldb/Utility/StructuredData.h" #include "lldb/Utility/UUID.h" #include "lldb/lldb-private.h" #include "llvm/Support/Threading.h" @@ -544,9 +545,9 @@ class ObjectFile : public std::enable_shared_from_this<ObjectFile>, return false; } - /// Get metadata about threads from the corefile. + /// Get metadata about thread ids from the corefile. /// - /// The corefile may have metadata (e.g. a Mach-O "thread extrainfo" + /// The corefile may have metadata (e.g. a Mach-O "process metadata" /// LC_NOTE) which for the threads in the process; this method tries /// to retrieve them. /// @@ -568,6 +569,18 @@ class ObjectFile : public std::enable_shared_from_this<ObjectFile>, return false; } + /// Get process metadata from the corefile in a StructuredData dictionary. + /// + /// The corefile may have notes (e.g. a Mach-O "process metadata" LC_NOTE) + /// which provide metadata about the process and threads in a JSON or + /// similar format. + /// + /// \return + /// A StructuredData object with the metadata in the note, if there is + /// one. An empty shared pointer is returned if not metadata is found, + /// or a problem parsing it. + virtual StructuredData::ObjectSP GetCorefileProcessMetadata() { return {}; } + virtual lldb::RegisterContextSP GetThreadContextAtIndex(uint32_t idx, lldb_private::Thread &thread) { return lldb::RegisterContextSP(); diff --git a/lldb/source/Plugins/ObjectFile/Mach-O/ObjectFileMachO.cpp b/lldb/source/Plugins/ObjectFile/Mach-O/ObjectFileMachO.cpp index b1741926b74aa..aea45293095e7 100644 --- a/lldb/source/Plugins/ObjectFile/Mach-O/ObjectFileMachO.cpp +++ b/lldb/source/Plugins/ObjectFile/Mach-O/ObjectFileMachO.cpp @@ -5794,27 +5794,9 @@ bool ObjectFileMachO::GetCorefileThreadExtraInfos( std::lock_guard<std::recursive_mutex> guard(module_sp->GetMutex()); Log *log(GetLog(LLDBLog::Object | LLDBLog::Process | LLDBLog::Thread)); - auto lc_notes = FindLC_NOTEByName("process metadata"); - for (auto lc_note : lc_notes) { - offset_t payload_offset = std::get<0>(lc_note); - offset_t strsize = std::get<1>(lc_note); - std::string buf(strsize, '\0'); - if (m_data.CopyData(payload_offset, strsize, buf.data()) != strsize) { - LLDB_LOGF(log, - "Unable to read %" PRIu64 - " bytes of 'process metadata' LC_NOTE JSON contents", - strsize); - return false; - } - while (buf.back() == '\0') - buf.resize(buf.size() - 1); - StructuredData::ObjectSP object_sp = StructuredData::ParseJSON(buf); + StructuredData::ObjectSP object_sp = GetCorefileProcessMetadata(); + if (object_sp) { StructuredData::Dictionary *dict = object_sp->GetAsDictionary(); - if (!dict) { - LLDB_LOGF(log, "Unable to read 'process metadata' LC_NOTE, did not " - "get a dictionary."); - return false; - } StructuredData::Array *threads; if (!dict->GetValueForKeyAsArray("threads", threads) || !threads) { LLDB_LOGF(log, @@ -5857,6 +5839,45 @@ bool ObjectFileMachO::GetCorefileThreadExtraInfos( return false; } +StructuredData::ObjectSP ObjectFileMachO::GetCorefileProcessMetadata() { + ModuleSP module_sp(GetModule()); + if (!module_sp) + return {}; + + Log *log(GetLog(LLDBLog::Object | LLDBLog::Process | LLDBLog::Thread)); + std::lock_guard<std::recursive_mutex> guard(module_sp->GetMutex()); + auto lc_notes = FindLC_NOTEByName("process metadata"); + if (lc_notes.size() == 0) + return {}; + + if (lc_notes.size() > 1) + LLDB_LOGF( + log, + "Multiple 'process metadata' LC_NOTEs found, only using the first."); + + offset_t payload_offset = std::get<0>(lc_notes[0]); + offset_t strsize = std::get<1>(lc_notes[0]); + std::string buf(strsize, '\0'); + if (m_data.CopyData(payload_offset, strsize, buf.data()) != strsize) { + LLDB_LOGF(log, + "Unable to read %" PRIu64 + " bytes of 'process metadata' LC_NOTE JSON contents", + strsize); + return {}; + } + while (buf.back() == '\0') + buf.resize(buf.size() - 1); + StructuredData::ObjectSP object_sp = StructuredData::ParseJSON(buf); + StructuredData::Dictionary *dict = object_sp->GetAsDictionary(); + if (!dict) { + LLDB_LOGF(log, "Unable to read 'process metadata' LC_NOTE, did not " + "get a dictionary."); + return {}; + } + + return object_sp; +} + lldb::RegisterContextSP ObjectFileMachO::GetThreadContextAtIndex(uint32_t idx, lldb_private::Thread &thread) { diff --git a/lldb/source/Plugins/ObjectFile/Mach-O/ObjectFileMachO.h b/lldb/source/Plugins/ObjectFile/Mach-O/ObjectFileMachO.h index 7f67f5e04f1d6..7e3a6754dd0b8 100644 --- a/lldb/source/Plugins/ObjectFile/Mach-O/ObjectFileMachO.h +++ b/lldb/source/Plugins/ObjectFile/Mach-O/ObjectFileMachO.h @@ -133,6 +133,8 @@ class ObjectFileMachO : public lldb_private::ObjectFile { bool GetCorefileThreadExtraInfos(std::vector<lldb::tid_t> &tids) override; + lldb_private::StructuredData::ObjectSP GetCorefileProcessMetadata() override; + bool LoadCoreFileImages(lldb_private::Process &process) override; lldb::RegisterContextSP diff --git a/lldb/source/Plugins/Process/mach-core/CMakeLists.txt b/lldb/source/Plugins/Process/mach-core/CMakeLists.txt index a1ea85ec4c728..926f06d23c64b 100644 --- a/lldb/source/Plugins/Process/mach-core/CMakeLists.txt +++ b/lldb/source/Plugins/Process/mach-core/CMakeLists.txt @@ -1,6 +1,7 @@ add_lldb_library(lldbPluginProcessMachCore PLUGIN ProcessMachCore.cpp ThreadMachCore.cpp + RegisterContextUnifiedCore.cpp LINK_COMPONENTS Support diff --git a/lldb/source/Plugins/Process/mach-core/RegisterContextUnifiedCore.cpp b/lldb/source/Plugins/Process/mach-core/RegisterContextUnifiedCore.cpp new file mode 100644 index 0000000000000..774c0ca605b3c --- /dev/null +++ b/lldb/source/Plugins/Process/mach-core/RegisterContextUnifiedCore.cpp @@ -0,0 +1,293 @@ +//===-- RegisterContextUnifiedCore.cpp ------------------------------------===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +#include "RegisterContextUnifiedCore.h" +#include "lldb/Target/DynamicRegisterInfo.h" +#include "lldb/Target/Process.h" +#include "lldb/Utility/DataExtractor.h" +#include "lldb/Utility/RegisterValue.h" +#include "lldb/Utility/StructuredData.h" + +using namespace lldb; +using namespace lldb_private; + +RegisterContextUnifiedCore::RegisterContextUnifiedCore( + Thread &thread, uint32_t concrete_frame_idx, + RegisterContextSP core_thread_regctx_sp, + StructuredData::ObjectSP metadata_thread_registers) + : RegisterContext(thread, concrete_frame_idx) { + + ProcessSP process_sp(thread.GetProcess()); + Target &target = process_sp->GetTarget(); + StructuredData::Dictionary *metadata_registers_dict = nullptr; + + // If we have thread metadata, check if the keys for register + // definitions are present; if not, clear the ObjectSP. + if (metadata_thread_registers && + metadata_thread_registers->GetAsDictionary()->HasKey("register_info")) { + metadata_registers_dict = metadata_thread_registers->GetAsDictionary() + ->GetValueForKey("register_info") + ->GetAsDictionary(); + if (metadata_registers_dict) + if (!metadata_registers_dict->HasKey("sets") || + !metadata_registers_dict->HasKey("registers")) + metadata_registers_dict = nullptr; + } + + // When creating a register set list from the two sources, + // the LC_THREAD aka core_thread_regctx_sp register sets + // will be used at the same indexes. + // Any additional sets named by the thread metadata registers + // will be added. If the thread metadata registers specify + // a set with the same name, the already-used index from the + // core register context will be used. + std::map<size_t, size_t> metadata_regset_to_combined_regset; + + // Calculate the total size of the register store buffer we need + // for all registers. The corefile register definitions may include + // RegisterInfo descriptions of registers that aren't actually + // available. For simplicity, calculate the size of all registers + // as if they are available, so we can maintain the same offsets into + // the buffer. + uint32_t core_buffer_end = 0; + for (size_t idx = 0; idx < core_thread_regctx_sp->GetRegisterCount(); idx++) { + const RegisterInfo *reginfo = + core_thread_regctx_sp->GetRegisterInfoAtIndex(idx); + core_buffer_end = + std::max(reginfo->byte_offset + reginfo->byte_size, core_buffer_end); + } + + // Add metadata register sizes to the total buffer size. + uint32_t combined_buffer_end = core_buffer_end; + if (metadata_registers_dict) { + StructuredData::Array *registers = nullptr; + if (metadata_registers_dict->GetValueForKeyAsArray("registers", registers)) + registers->ForEach( + [&combined_buffer_end](StructuredData::Object *ent) -> bool { + uint32_t bitsize; + if (!ent->GetAsDictionary()->GetValueForKeyAsInteger("bitsize", + bitsize)) + return false; + combined_buffer_end += (bitsize / 8); + return true; + }); + } + m_register_data.resize(combined_buffer_end, 0); + + // Copy the core register values into our combined data buffer, + // skip registers that are contained within another (e.g. w0 vs. x0) + // and registers that return as "unavailable". + for (size_t idx = 0; idx < core_thread_regctx_sp->GetRegisterCount(); idx++) { + const RegisterInfo *reginfo = + core_thread_regctx_sp->GetRegisterInfoAtIndex(idx); + RegisterValue val; + if (!reginfo->value_regs && + core_thread_regctx_sp->ReadRegister(reginfo, val)) + memcpy(m_register_data.data() + reginfo->byte_offset, val.GetBytes(), + val.GetByteSize()); + } + + // Set 'offset' fields for each register definition into our combined + // register data buffer. DynamicRegisterInfo needs + // this field set to parse the JSON. + // Also copy the values of the registers into our register data buffer. + if (metadata_registers_dict) { + size_t offset = core_buffer_end; + ByteOrder byte_order = core_thread_regctx_sp->GetByteOrder(); + StructuredData::Array *registers; + if (metadata_registers_dict->GetValueForKeyAsArray("registers", registers)) + registers->ForEach([this, &offset, + byte_order](StructuredData::Object *ent) -> bool { + uint64_t bitsize; + uint64_t value; + if (!ent->GetAsDictionary()->GetValueForKeyAsInteger("bitsize", + bitsize)) + return false; + if (!ent->GetAsDictionary()->GetValueForKeyAsInteger("value", value)) { + // we had a bitsize but no value, so move the offset forward I guess. + offset += (bitsize / 8); + return false; + } + ent->GetAsDictionary()->AddIntegerItem("offset", offset); + Status error; + switch (bitsize / 8) { + case 2: { + Scalar value_scalar((uint16_t)value); + value_scalar.GetAsMemoryData(m_register_data.data() + offset, 2, + byte_order, error); + offset += 2; + } break; + case 4: { + Scalar value_scalar((uint32_t)value); + value_scalar.GetAsMemoryData(m_register_data.data() + offset, 4, + byte_order, error); + offset += 4; + } break; + case 8: { + Scalar value_scalar((uint64_t)value); + value_scalar.GetAsMemoryData(m_register_data.data() + offset, 8, + byte_order, error); + offset += 8; + } break; + } + return true; + }); + } + + // Create a DynamicRegisterInfo from the metadata JSON. + std::unique_ptr<DynamicRegisterInfo> additional_reginfo_up; + if (metadata_registers_dict) + additional_reginfo_up = DynamicRegisterInfo::Create( + *metadata_registers_dict, target.GetArchitecture()); + + // Copy the core thread register sets into our combined register set list. + for (size_t idx = 0; idx < core_thread_regctx_sp->GetRegisterSetCount(); + idx++) { + RegisterSet new_set; + const RegisterSet *old_set = core_thread_regctx_sp->GetRegisterSet(idx); + new_set.name = ConstString(old_set->name).AsCString(); + if (old_set->short_name) + new_set.short_name = ConstString(old_set->short_name).AsCString(); + else + new_set.short_name = nullptr; + m_register_sets.push_back(new_set); + } + + // Set up our RegisterSet array. + // First copying all of the core thread register sets, + // then any additional unique register sets from the metadata. + if (additional_reginfo_up) { + for (size_t idx = 0; idx < additional_reginfo_up->GetNumRegisterSets(); + idx++) { + bool found_match = false; + const RegisterSet *old_set = additional_reginfo_up->GetRegisterSet(idx); + for (size_t jdx = 0; jdx < m_register_sets.size(); jdx++) { + if (strcmp(m_register_sets[jdx].name, old_set->name) == 0) { + metadata_regset_to_combined_regset[idx] = jdx; + found_match = true; + break; + } + } + if (!found_match) { + RegisterSet new_set; + new_set.name = ConstString(old_set->name).AsCString(); + if (old_set->short_name) + new_set.short_name = ConstString(old_set->short_name).AsCString(); + else + new_set.short_name = nullptr; + metadata_regset_to_combined_regset[idx] = m_register_sets.size(); + m_register_sets.push_back(new_set); + } + } + } + + // Set up our RegisterInfo array, one RegisterSet at a time. + // The metadata registers may be declared to be in a core thread + // register set (e.g. "General Purpose Registers", so we scan + // both core registers and metadata registers should be examined + // when creating the combined register sets. + for (size_t combined_regset_idx = 0; + combined_regset_idx < m_register_sets.size(); combined_regset_idx++) { + uint32_t registers_this_regset = 0; + if (combined_regset_idx < core_thread_regctx_sp->GetRegisterSetCount()) { + const RegisterSet *regset = + core_thread_regctx_sp->GetRegisterSet(combined_regset_idx); + for (size_t j = 0; j < regset->num_registers; j++) { + uint32_t reg_idx = regset->registers[j]; + const RegisterInfo *reginfo = + core_thread_regctx_sp->GetRegisterInfoAtIndex(reg_idx); + RegisterValue val; + if (!reginfo->value_regs && + core_thread_regctx_sp->ReadRegister(reginfo, val)) { + m_regset_regnum_collection[combined_regset_idx].push_back( + m_register_infos.size()); + m_register_infos.push_back(*reginfo); + registers_this_regset++; + } + } + } + if (additional_reginfo_up) { + // Find the register set in the metadata that matches this register + // set, then copy all its RegisterInfos. + for (size_t setidx = 0; + setidx < additional_reginfo_up->GetNumRegisterSets(); setidx++) { + if (metadata_regset_to_combined_regset[setidx] == combined_regset_idx) { + const RegisterSet *regset = + additional_reginfo_up->GetRegisterSet(setidx); + for (size_t j = 0; j < regset->num_registers; j++) { + uint32_t reg_idx = regset->registers[j]; + const RegisterInfo *reginfo = + additional_reginfo_up->GetRegisterInfoAtIndex(reg_idx); + m_regset_regnum_collection[combined_regset_idx].push_back( + m_register_infos.size()); + // register names in DynamicRegisterInfo are ConstString stored; + // we can reuse the char* pointers here without retaining the + // DynamicRegisterInfo. + m_register_infos.push_back(*reginfo); + registers_this_regset++; + } + } + } + } + m_register_sets[combined_regset_idx].num_registers = registers_this_regset; + m_register_sets[combined_regset_idx].registers = + m_regset_regnum_collection[combined_regset_idx].data(); + } +} + +size_t RegisterContextUnifiedCore::GetRegisterCount() { + return m_register_infos.size(); +} + +const RegisterInfo * +RegisterContextUnifiedCore::GetRegisterInfoAtIndex(size_t reg) { + return &m_register_infos[reg]; +} + +size_t RegisterContextUnifiedCore::GetRegisterSetCount() { + return m_register_sets.size(); +} + +const RegisterSet *RegisterContextUnifiedCore::GetRegisterSet(size_t set) { + return &m_register_sets[set]; +} + +bool RegisterContextUnifiedCore::ReadRegister( + const lldb_private::RegisterInfo *reg_info, + lldb_private::RegisterValue &value) { + if (!reg_info) + return false; + ProcessSP process_sp(m_thread.GetProcess()); + if (process_sp) { + DataExtractor regdata(m_register_data.data(), m_register_data.size(), + process_sp->GetByteOrder(), + process_sp->GetAddressByteSize()); + offset_t offset = reg_info->byte_offset; + switch (reg_info->byte_size) { + case 2: + value.SetUInt16(regdata.GetU16(&offset)); + break; + case 4: + value.SetUInt32(regdata.GetU32(&offset)); + break; + case 8: + value.SetUInt64(regdata.GetU64(&offset)); + break; + default: + return false; + } + return true; + } + return false; +} + +bool RegisterContextUnifiedCore::WriteRegister( + const lldb_private::RegisterInfo *reg_info, + const lldb_private::RegisterValue &value) { + return false; +} diff --git a/lldb/source/Plugins/Process/mach-core/RegisterContextUnifiedCore.h b/lldb/source/Plugins/Process/mach-core/RegisterContextUnifiedCore.h new file mode 100644 index 0000000000000..47e2fb7cac5a3 --- /dev/null +++ b/lldb/source/Plugins/Process/mach-core/RegisterContextUnifiedCore.h @@ -0,0 +1,57 @@ +//===-- RegisterContextUnifiedCore.h ----------------------------*- C++ -*-===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// + +#ifndef LLDB_SOURCE_PLUGINS_PROCESS_REGISTERCONTEXT_UNIFIED_CORE_H +#define LLDB_SOURCE_PLUGINS_PROCESS_REGISTERCONTEXT_UNIFIED_CORE_H + +#include <string> +#include <vector> + +#include "lldb/Target/RegisterContext.h" +#include "lldb/Utility/ConstString.h" +#include "lldb/Utility/StructuredData.h" +#include "lldb/lldb-enumerations.h" +#include "lldb/lldb-private.h" + +namespace lldb_private { + +class RegisterContextUnifiedCore : public RegisterContext { +public: + RegisterContextUnifiedCore( + Thread &thread, uint32_t concrete_frame_idx, + lldb::RegisterContextSP core_thread_regctx_sp, + lldb_private::StructuredData::ObjectSP metadata_thread_registers); + + void InvalidateAllRegisters() override{}; + + size_t GetRegisterCount() override; + + const lldb_private::RegisterInfo *GetRegisterInfoAtIndex(size_t reg) override; + + size_t GetRegisterSetCount() override; + + const lldb_private::RegisterSet *GetRegisterSet(size_t set) override; + + bool ReadRegister(const lldb_private::RegisterInfo *reg_info, + lldb_private::RegisterValue &value) override; + + bool WriteRegister(const lldb_private::RegisterInfo *reg_info, + const lldb_private::RegisterValue &value... [truncated] `````````` </details> https://github.com/llvm/llvm-project/pull/144627 _______________________________________________ lldb-commits mailing list lldb-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
