Android ART VM的文件格式－OAT（四）

我就是我 2022-05-31 08:36 248阅读 0赞

一，与oat相关的文件后缀

参考老罗的博客：[OAT文件][OAT]

１）.oat，OAT是由dex2oat产生的，本质上也是属于elf文件。

２）.odex，在Dalvik中，odex表示被优化后的dex文件；ART虚拟机中，它实际上是oat文件。

oat文件除了遵循elf文件规范，又根据虚拟机的需求进行了扩展－－最大的区别增加了两个重要的字段 oat data section 和oat exec section，其中data section保存的是原dex文件中的字节码数据，exec section是dex经过dex2oat编译后生成的机器码的存储区域。可以从data section中通过一定的对应关系可以迅速找到某个class/function在exec section中机器码。

加载oat文件的函数是dlopen。

art/runtime/oat\_file.cc

bool DlOpenOatFile::Dlopen(const std::string& elf_filename,
                               uint8_t* oat_file_begin,
                               std::string* error_msg) {
    #ifdef __APPLE__
    #else
      {
        UniqueCPtr<char> absolute_path(realpath(elf_filename.c_str(), nullptr));
        if (absolute_path == nullptr) {
          *error_msg = StringPrintf("Failed to find absolute path for '%s'", elf_filename.c_str());
          return false;
        }
    #ifdef __ANDROID__
        android_dlextinfo extinfo;
        extinfo.flags = ANDROID_DLEXT_FORCE_LOAD |                  // Force-load, don't reuse handle
                                                                    //   (open oat files multiple
                                                                    //    times).
                        ANDROID_DLEXT_FORCE_FIXED_VADDR;            // Take a non-zero vaddr as absolute
                                                                    //   (non-pic boot image).
        if (oat_file_begin != nullptr) {                            //
          extinfo.flags |= ANDROID_DLEXT_LOAD_AT_FIXED_ADDRESS;     // Use the requested addr if
          extinfo.reserved_addr = oat_file_begin;                   // vaddr = 0.
        }                                                           //   (pic boot image).
        dlopen_handle_ = android_dlopen_ext(absolute_path.get(), RTLD_NOW, &extinfo);
    #else
    #endif
      }
      if (dlopen_handle_ == nullptr) {
        *error_msg = StringPrintf("Failed to dlopen '%s': %s", elf_filename.c_str(), dlerror());
        return false;
      }
      return true;
    #endif
    }

OAT文件的内部结构，可以拜读老罗的博客。

二，OAT的编译时机

１）Rom构建的时候，

/build/core/Dex\_preopt.mk

include $(BUILD_SYSTEM)/dex_preopt_libart.mk
    
    # Define dexpreopt-one-file based on current default runtime.
    # $(1): the input .jar or .apk file
    # $(2): the output .odex file
    define dexpreopt-one-file
    $(call dex2oat-one-file,$(1),$(2))
    endef

在系统构建时会执行dex2oat编译。art程序的预优化脚本是dex\_preopt\_libart.mk。

dex2oat是在编译的那个阶段执行的优化操作呢？实际是在需要执行优化的所有模块（包括jar,apk），在其产生dex文件的地方，开始执行优化的，具体就是调用dexpreopt-one-file脚本，为dex2oat的执行提供上下文环境。

dex2oat程序支持的选项参数：

\-j <number> :指定执行编译操作所需的线程数量。

\--dex-file=<dex-file> 输入参数: 指定需要被编译的.dex, .jar,或apk文件（内含.dex），如：--dex-file=/system/framework/core.jar

\--oat-file=<file.oat> 输出参数：以文件名的形式指定oat的输出目标。如：--oat-file=/system/framework/boot.oat

\--oat-symbols=<file.oat> 输出参数：指定带有完全符号表的OAT输出目标，如：--oat-file=/symbols/system/framework/boot.oat

......

２）第三方应用安装的时候，执行dex2oat，在点击安装apk文件时，是怎样关联到Android系统的安装服务的？

应用安装的起点是startActivity的调用，响应安装请求的是packageinstallerActivity，然后调用InstallAppProcess进入下一步的操作，InstallAppProcess一方面显示安装进度，一方面调用系统服务packageManagerservice执行具体的安装过程。

PMS在安装apk的过程中，会跟installd（init.rc中启动的daemon进程）这个系统服务通信。PMS中的变量mInstaller是PMS跟installd沟通的桥梁，Installer基于socket来建立跟installd的通信链接。

如果PMS判断要安装的apk需要dex2oat编译，会调用Installer的函数dexopt：

framework/base/services/core/java/com/android/server/pm/Installer.java

public void dexopt(String apkPath, int uid, String instructionSet, int dexoptNeeded,
                int dexFlags, String compilerFilter, String volumeUuid, String sharedLibraries)
                throws InstallerException {
            assertValidInstructionSet(instructionSet);
            mInstaller.dexopt(apkPath, uid, instructionSet, dexoptNeeded, dexFlags,
                    compilerFilter, volumeUuid, sharedLibraries);
        }

这个函数中mInstaller指的是InstallerConnection实例，也就是连接installd的一个通道。

framework/base/core/java/com/android/internal/os/InstallerConnection.java

public void dexopt(String apkPath, int uid, String pkgName, String instructionSet,
                int dexoptNeeded, String outputPath, int dexFlags, String compilerFilter,
                String volumeUuid, String sharedLibraries) throws InstallerException {
            execute("dexopt",
                    apkPath,
                    uid,
                    pkgName,
                    instructionSet,
                    dexoptNeeded,
                    outputPath,
                    dexFlags,
                    compilerFilter,
                    volumeUuid,
                    sharedLibraries);
        }

先构造传递到installd的参数，然后通过execute执行，真正往installd发送命令是通过transact函数。

public synchronized String transact(String cmd) {
    //如果连接还没建立，要先去执行连接
            if (!connect()) {
                Slog.e(TAG, "connection failed");
                return "-1";
            }
    //向installd写入命令
            if (!writeCommand(cmd)) {
                /*
                 * If installd died and restarted in the background (unlikely but
                 * possible) we'll fail on the next write (this one). Try to
                 * reconnect and write the command one more time before giving up.
                 */
    　　　　　　　//写入失败，再次尝试连接、写入数据
    　　　　　　　Slog.e(TAG, "write command failed? reconnect!");
                if (!connect() || !writeCommand(cmd)) {
                    return "-1";
                }
            }
    //读取installd的执行结果
            final int replyLength = readReply();
            if (replyLength > 0) {
                String s = new String(buf, 0, replyLength);
                return s;
            } else {
                return "-1";
            }
        }

这样命令就从PMS发送到installd了，installd作为一个daemon进程，启动后会在一个循环中等待连接，然后接收命令，处理命令。

frameworks/native/cmds/installd/installd.cpp

static int execute(int s, char cmd[BUFFER_MAX])
    {
        char reply[REPLY_MAX];
        char *arg[TOKEN_MAX+1];
        unsigned i;
        unsigned n = 0;
        unsigned short count;
        int ret = -1;
            /* default reply is "" */
        reply[0] = 0;
    
            /* n is number of args (not counting arg[0]) */
        arg[0] = cmd;
        while (*cmd) {
        //循环处理所有参数，多个参数之间是以空格做分隔符的，跟发送端的格式一致
             if (isspace(*cmd)) {
                *cmd++ = 0;
                n++;
                arg[n] = cmd;
                if (n == TOKEN_MAX) {
                    ALOGE("too many arguments\n");
                    goto done;
                }
            }
            if (*cmd) {
              cmd++;
            }
        }
    //cmds是一个数组，定义了命令跟处理函数之间的映射
        for (i = 0; i < sizeof(cmds) / sizeof(cmds[0]); i++) {
            if (!strcmp(cmds[i].name,arg[0])) {
                if (n != cmds[i].numargs) {
                    ALOGE("%s requires %d arguments (%d given)\n",
                         cmds[i].name, cmds[i].numargs, n);
                } else {
                    ret = cmds[i].func(arg + 1, reply);
                }
                goto done;
            }
        }
        ALOGE("unsupported command '%s'\n", arg[0]);
    
    done:
        if (reply[0]) {
            n = snprintf(cmd, BUFFER_MAX, "%d %s", ret, reply);
        } else {
            n = snprintf(cmd, BUFFER_MAX, "%d", ret);
        }
        if (n > BUFFER_MAX) n = BUFFER_MAX;
        count = n;
    
        // ALOGI("reply: '%s'\n", cmd);
        if (writex(s, &count, sizeof(count))) return -1;
        if (writex(s, cmd, count)) return -1;
        return 0;
    }

这个数组是commond跟执行函数的映射关系：

struct cmdinfo cmds[] = {
        { "ping",                 0, do_ping },
    
        { "create_app_data",      7, do_create_app_data },
        { "restorecon_app_data",  6, do_restorecon_app_data },
        { "migrate_app_data",     4, do_migrate_app_data },
        { "clear_app_data",       5, do_clear_app_data },
        { "destroy_app_data",     5, do_destroy_app_data },
        { "move_complete_app",    7, do_move_complete_app },
        { "get_app_size",         6, do_get_app_size },
        { "get_app_data_inode",   4, do_get_app_data_inode },
    
        { "create_user_data",     4, do_create_user_data },
        { "destroy_user_data",    3, do_destroy_user_data },
    
        { "dexopt",              10, do_dexopt },
        { "markbootcomplete",     1, do_mark_boot_complete },
        { "rmdex",                2, do_rm_dex },
        { "freecache",            2, do_free_cache },
        { "linklib",              4, do_linklib },
        { "idmap",                3, do_idmap },
        { "createoatdir",         2, do_create_oat_dir },
        { "rmpackagedir",         1, do_rm_package_dir },
        { "clear_app_profiles",   1, do_clear_app_profiles },
        { "destroy_app_profiles", 1, do_destroy_app_profiles },
        { "linkfile",             3, do_link_file },
        { "move_ab",              3, do_move_ab },
        { "merge_profiles",       2, do_merge_profiles },
        { "dump_profiles",        3, do_dump_profiles },
        { "delete_odex",          3, do_delete_odex },
    };

可以看到dexopt对应是do\_dexopt函数。

static int do_dexopt(char **arg, char reply[REPLY_MAX])
    {
        const char* args[DEXOPT_PARAM_COUNT];
        for (size_t i = 0; i < DEXOPT_PARAM_COUNT; ++i) {
            CHECK(arg[i] != nullptr);
            args[i] = arg[i];
        }
    
        int dexopt_flags = atoi(arg[6]);
        DexoptFn dexopt_fn;
        if ((dexopt_flags & DEXOPT_OTA) != 0) {
            dexopt_fn = do_ota_dexopt;
        } else {
            dexopt_fn = do_regular_dexopt;
        }
        return dexopt_fn(args, reply);
    }

以普通应用安装，是执行do\_regular\_dexopt，最终的实现是调用了Commonds.cpp中的dexopt函数：

frameworks/native/cmds/installd/Commonds.cpp

int dexopt(const char* apk_path, uid_t uid, const char* pkgname, const char* instruction_set,
               int dexopt_needed, const char* oat_dir, int dexopt_flags, const char* compiler_filter,
               const char* volume_uuid ATTRIBUTE_UNUSED, const char* shared_libraries)
    {
        bool is_public = ((dexopt_flags & DEXOPT_PUBLIC) != 0);
        bool vm_safe_mode = (dexopt_flags & DEXOPT_SAFEMODE) != 0;
        bool debuggable = (dexopt_flags & DEXOPT_DEBUGGABLE) != 0;
        bool boot_complete = (dexopt_flags & DEXOPT_BOOTCOMPLETE) != 0;
        bool profile_guided = (dexopt_flags & DEXOPT_PROFILE_GUIDED) != 0;
    
        char out_path[PKG_PATH_MAX];
    //这里会创建dalvik-cache文件
         if (!create_oat_out_path(apk_path, instruction_set, oat_dir, out_path)) {
            return false;
        }
    
        const char *input_file;
        char in_odex_path[PKG_PATH_MAX];
        switch (dexopt_needed) {
            case DEXOPT_DEX2OAT_NEEDED:
                input_file = apk_path;
                break;
    
            case DEXOPT_PATCHOAT_NEEDED:
                if (!calculate_odex_file_path(in_odex_path, apk_path, instruction_set)) {
                    return -1;
                }
                input_file = in_odex_path;
                break;
    
            case DEXOPT_SELF_PATCHOAT_NEEDED:
                input_file = out_path;
                break;
    
            default:
                ALOGE("Invalid dexopt needed: %d\n", dexopt_needed);
                return 72;
        }
    
        struct stat input_stat;
        memset(&input_stat, 0, sizeof(input_stat));
        stat(input_file, &input_stat);
    //打开输入文件，也即是apk文件
        base::unique_fd input_fd(open(input_file, O_RDONLY, 0));
        if (input_fd.get() < 0) {
            ALOGE("installd cannot open '%s' for input during dexopt\n", input_file);
            return -1;
        }
    
        const std::string out_path_str(out_path);
    //创建输出结果文件
         Dex2oatFileWrapper<std::function<void ()>> out_fd(
                open_output_file(out_path, /*recreate*/true, /*permissions*/0644),
                [out_path_str]() { unlink(out_path_str.c_str()); });
        if (out_fd.get() < 0) {
            ALOGE("installd cannot open '%s' for output during dexopt\n", out_path);
            return -1;
        }
        if (!set_permissions_and_ownership(out_fd.get(), is_public, uid, out_path)) {
            return -1;
        }
        ALOGV("DexInv: --- BEGIN '%s' ---\n", input_file);
    //启动一个子进程，来真正执行dex2oat的编译，因为dex2oat是一个独立的程序，所以为它的运行创建一个新的进程
        pid_t pid = fork();
        if (pid == 0) {
            /* child -- drop privileges before continuing */
            drop_capabilities(uid);
    
            SetDex2OatAndPatchOatScheduling(boot_complete);
            if (flock(out_fd.get(), LOCK_EX | LOCK_NB) != 0) {
                ALOGE("flock(%s) failed: %s\n", out_path, strerror(errno));
                _exit(67);
            }
    
            if (dexopt_needed == DEXOPT_PATCHOAT_NEEDED
                || dexopt_needed == DEXOPT_SELF_PATCHOAT_NEEDED) {
                run_patchoat(input_fd.get(),
                             out_fd.get(),
                             input_file,
                             out_path,
                             pkgname,
                             instruction_set);
            } else if (dexopt_needed == DEXOPT_DEX2OAT_NEEDED) {
                // Pass dex2oat the relative path to the input file.
                const char *input_file_name = get_location_from_path(input_file);
            //这个调用是实际执行编译的地方
                run_dex2oat(input_fd.get(),
                            out_fd.get(),
                            image_fd.get(),
                            input_file_name,
                            out_path,
                            swap_fd.get(),
                            instruction_set,
                            compiler_filter,
                            vm_safe_mode,
                            debuggable,
                            boot_complete,
                            reference_profile_fd.get(),
                            shared_libraries);
            } else {
                ALOGE("Invalid dexopt needed: %d\n", dexopt_needed);
                _exit(73);
            }
            _exit(68);   /* only get here on exec failure */
        } else {
    //父进程要等着子进程任务完成。
             int res = wait_child(pid);
            if (res == 0) {
                ALOGV("DexInv: --- END '%s' (success) ---\n", input_file);
            } else {
                ALOGE("DexInv: --- END '%s' --- status=0x%04x, process failed\n", input_file, res);
                return -1;
            }
        }
    
        struct utimbuf ut;
        ut.actime = input_stat.st_atime;
        ut.modtime = input_stat.st_mtime;
        utime(out_path, &ut);
    
        // We've been successful, don't delete output.
        out_fd.SetCleanup(false);
        image_fd.SetCleanup(false);
        reference_profile_fd.SetCleanup(false);
    
        return 0;
    }

到这里，第三方apk的dex2oat的编译就完成了。

[OAT]: http://blog.csdn.net/luoshengyang/article/details/39307813