关于故障硬盘数据迁移的粗略研究——硬盘信息采集

  1. 1. 简介
  2. 2. 实现步骤
    1. 2.1. 一、在 Java 程序中调用 Windows 系统命令
    2. 2.2. 二、通过 PowerShell 的 Get-WmiObject 命令获取硬盘信息

简介

  在该项目中,我们提供了集群监控、硬盘故障预测的实现,甲方进一步提出要能在预测出故障后方便地进行磁盘数据转移。由于之前没研究过数据迁移和故障恢复,所以只能从头慢慢摸索。目前的大致思路就是:将源硬盘(快故障的硬盘)的数据进行压缩;将目的硬盘(正常硬盘)格式化并进行分区;在源节点与目的节点之间进行“点对点”的数据传输;最后在目的硬盘上恢复数据。项目地址在这里

  目前仅在 Windows 系统上进行了实现,Linux 系统上由于 dd 、scp 等命令的存在,可能实现起来更加简单。

实现步骤

一、在 Java 程序中调用 Windows 系统命令

  • 执行系统命令

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    // ExeCmd.java
    public static void executeCmd(String command[], String path, StringBuilder infoList, StringBuilder errorList) {
    Runtime runtime = Runtime.getRuntime(); // 每个 Java 应用程序都有一个 Runtime 类的单例,它允许应用程序与应用程序运行的环境进行交互
    Process process = null; // 在单独的进程中执行指定的命令和参数。
    try {
    if (path == null)
    process = runtime.exec(command);
    else
    process = runtime.exec(command, null, new File(path));

    InputStream info = process.getInputStream();
    InputStream err = process.getErrorStream();
    new Thread(new InputStreamThread(info, infoList)).start(); // 获取执行结果
    new Thread(new InputStreamThread(err, errorList)).start(); // 获取错误信息
    process.waitFor(); // 等待所有子线程执行完毕
    } catch (IOException e) {
    logger.error("[executeCmd][命令执行错误][cmd :{}{}]error info: {}", path, Arrays.toString(command), e.toString());
    } catch (InterruptedException e) {
    logger.error("[executeCmd][命令中断][cmd :{}{}]error info: {}", path, Arrays.toString(command), e.toString());
    e.printStackTrace();
    } finally {
    try {
    process.getOutputStream().close();
    } catch (Exception e) {
    logger.error("[executeCmd]error info: {}", e.toString());
    }
    }
    }
  • 获取返回信息

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    static class InputStreamThread implements Runnable {
    private BufferedReader bufferedReader = null;
    private InputStream inputStream = null;
    private StringBuilder info = null; // 通过 StringBuilder 捕获打印信息,也可以用其他方式

    public InputStreamThread(InputStream inputStream, StringBuilder info) {
    this.inputStream = inputStream;
    this.info = info;
    this.bufferedReader = new BufferedReader(new InputStreamReader(inputStream));
    }

    @Override
    public void run() {
    String line = null;
    try {
    while ((line = bufferedReader.readLine()) != null) {
    if (info != null)
    info.append(line);
    logger.info("[executeCmd]info: {}", line);
    }
    } catch (Exception e) {
    logger.error("[executeCmd]error: {}", e.toString());
    } finally {
    try {
    inputStream.close();
    } catch (Exception e) {
    logger.error("[executeCmd]error: {}", e.toString());
    }
    }
    }

    }

    executeCmd() 在后面会经常用到。

二、通过 PowerShell 的 Get-WmiObject 命令获取硬盘信息

  • 什么是 Wmi

    ​ WMI就是 Windows Management Instrumentation(Windows 管理规范)。它是 Windows 中的一个核心管理技术。WMI 为访问大量的 Windows 管理数据和方法的提供了一个统一的机制。WMI通过脚本、C++程序接口、.NET类(系统管理)和命令行工具(WMIC)提供了对这个信息的访问。WMI的功能还包括事件、远程、查询、查看、计划和实施用户扩展及更多内容。

    例如保存硬盘分区信息的 Wmi 类如下:(详细可查询

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    class Win32_DiskPartition : CIM_DiskPartition
    {
    unit16 AdditionalAvailability;
    uint16 Availability;
    uint16 PowerManagementCapabilities[];
    string IdentifyingDescriptions[1];
    uint64 MaxQuiesceTime;
    uint64 OtherIdentifyingInfo;
    uint16 StatusInfo;
    uint64 PowerOnHours;
    uint64 TotalPowerOnHours;
    uint16 Access;
    uint64 BlockSize;
    boolean Bootable;
    boolean BootPartition;
    string. Caption;
    uint32 ConfigManagerErrorCode;
    boolean ConfigManagerUserConfig;
    string. CreationClassName;
    string Description;
    string DeviceID;
    uint32 DiskIndex;
    boolean ErrorCleared;
    string ErrorDescription;
    string ErrorMethodology;
    uint32 HiddenSectors;
    uint32 Index;
    datetime InstallDate;
    uint32 LastErrorCode;
    string Name;
    uint64 NumberOfBlocks;
    string PNPDeviceID;
    boolean PowerManagementSupported;
    boolean PrimaryPartition;
    string Purpose;
    boolean RewritePartition;
    uint64 Size;
    uint64 StartingOffset;
    string Status;
    string SystemCreationClassName;
    string SystemName;
    string Type;
    };
  • 通过 PowerShell 命令获取Wmi信息

    这里要使用到的命令为 Get-WmiObject,命令格式为:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    Get-WmiObject
    [-Class] <String>
    [[-Property] <String[]>]
    [-Filter <String>]
    [-Amended]
    [-DirectRead]
    [-AsJob]
    [-Impersonation <ImpersonationLevel>]
    [-Authentication <AuthenticationLevel>]
    [-Locale <String>]
    [-EnableAllPrivileges]
    [-Authority <String>]
    [-Credential <PSCredential>]
    [-ThrottleLimit <Int32>]
    [-ComputerName <String[]>]
    [-Namespace <String>]
    [<CommonParameters>]

    我自己封装的获取硬盘信息的工具类如下:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    68
    69
    70
    71
    72
    73
    74
    75
    76
    77
    78
    79
    80
    81
    82
    83
    84
    85
    86
    87
    88
    89
    90
    91
    92
    93
    94
    95
    96
    97
    98
    99
    100
    101
    102
    103
    104
    public class CIMWin32_WMI {
    private final static String disk_info = "Index, SerialNumber, DeviceID, Caption, Size";
    private final static String wql_disk_info = String.format("SELECT %s FROM Win32_DiskDrive", disk_info);
    private final static String filter_disk_info = "Index, SerialNumber, DeviceID, Caption, Size";
    private final static String GET_DISK_INFO = String.format("Get-WmiObject -Query '%s' | Select-Object %s | ConvertTo-Json", wql_disk_info, filter_disk_info);


    private final static String partition_info = "DiskIndex, DeviceID, Type, Bootable, BootPartition, PrimaryPartition";
    private final static String wql_partition_info = String.format("SELECT %s FROM Win32_DiskPartition", partition_info);
    private final static String filter_partition_info = "DiskIndex, DeviceID, @{Name='Type'; Expression={if ($_.Type.StartsWith('GPT')){'GPT'}else{'MBR'}}}, Bootable, BootPartition, PrimaryPartition";
    private final static String GET_PARTITION_INFO = String.format("Get-WmiObject -Query '%s' | Select-Object %s | ConvertTo-Json", wql_partition_info, filter_partition_info);


    private final static String logical_disk_info = "Caption, Name, DeviceID, FileSystem, Size, FreeSpace";
    private final static String wql_logical_disk_info = String.format("SELECT %s FROM Win32_LogicalDisk", logical_disk_info);
    private final static String filter_logical_disk_info = "Caption, Name, DeviceID, FileSystem, Size, FreeSpace";
    private final static String GET_LOGICAL_DISK_INFO = String.format("Get-WmiObject -Query '%s' | Select-Object %s | ConvertTo-Json", wql_logical_disk_info, filter_logical_disk_info);


    private final static String ldisk_to_partition_info = "Antecedent, Dependent";
    private final static String wql_ldisk_to_partition_info = String.format("SELECT %s FROM Win32_LogicalDiskToPartition", ldisk_to_partition_info);
    private final static String filter_ldisk_to_partition_info = "Antecedent, Dependent";
    private final static String GET_lDISK_TO_PARTITION_INFO = String.format("Get-WmiObject -Query '%s' | Select-Object %s | ConvertTo-Json", wql_ldisk_to_partition_info, filter_ldisk_to_partition_info);

    /**
    * @name:
    * @desc: 获取原始硬盘信息并以JSON格式返回
    * @param {*}
    * @return {*}
    */
    public static JSONArray getDiskInfo() {

    JSONArray diskInfo = execWMI(GET_DISK_INFO);
    JSONArray partitionInfo = execWMI(GET_PARTITION_INFO);
    JSONArray logicalDiskInfo = execWMI(GET_LOGICAL_DISK_INFO);
    JSONArray ldiskToPartitionInfo = execWMI(GET_lDISK_TO_PARTITION_INFO);

    // 首先关联逻辑硬盘与分区信息
    ldiskToPartitionInfo.forEach(jsonObject -> {
    JSONObject relate = (JSONObject) jsonObject;
    String partitionDeviceId = relate.getString("Antecedent").split("=")[1].replace("\\", "").replace("\"", "");
    String logicalDiskDeviceId = relate.getString("Dependent").split("=")[1].replace("\\", "").replace("\"", "");

    partitionInfo.stream().anyMatch(jsonObject1 -> {
    JSONObject partition = (JSONObject) jsonObject1;
    if (partitionDeviceId.equalsIgnoreCase(partition.getString("DeviceID"))) {
    logicalDiskInfo.stream().anyMatch(jsonObject2 -> {
    JSONObject logical = (JSONObject) jsonObject2;
    if (logicalDiskDeviceId.equalsIgnoreCase(logical.getString("DeviceID"))) {
    logical.put("DiskIndex", partition.getIntValue("DiskIndex"));
    logical.put("Type", partition.getString("Type"));
    logical.put("BootPartition", partition.getBooleanValue("BootPartition"));
    logical.put("Bootable", partition.getBooleanValue("Bootable"));
    logical.put("PrimaryPartition", partition.getBooleanValue("PrimaryPartition"));
    return true;
    } else {
    return false;
    }
    });
    return true;
    } else {
    return false;
    }
    });
    });

    // 最后关联物理硬盘与逻辑硬盘
    diskInfo.forEach(jsonObject -> {
    JSONObject disk = (JSONObject) jsonObject;
    disk.put("Logical_Disks", new JSONArray());
    logicalDiskInfo.forEach(jsonObject1 -> {
    JSONObject logical = (JSONObject) jsonObject1;
    if (logical.containsKey("DiskIndex") && disk.getIntValue("Index") == logical.getIntValue("DiskIndex")){
    disk.put("Type", logical.getString("Type"));
    disk.getJSONArray("Logical_Disks").add(logical);
    }
    });
    });

    return diskInfo;
    }

    /**
    * @name:
    * @desc: 执行命令获取原始硬盘信息
    * @param {String} cmd
    * @return {*}
    */
    private static JSONArray execWMI(String cmd) {
    StringBuilder info = new StringBuilder();
    ExeCmd.executeCmd(new String[] {"powershell", cmd}, null, info, null);
    Object object = JSON.parse(info.toString());
    JSONArray res = null;
    if (object instanceof JSONObject) {
    res = new JSONArray();
    res.add(object);
    } else if (object instanceof JSONArray) {
    res = (JSONArray)object;
    } else {
    System.out.println("未知类型");
    }
    return res;
    }
    }

    至此,我们已经能获取到硬盘的基本信息了,在后续的工作中,需要基于这些信息进行硬盘数据转移工作,包括硬盘分区、数据传输、数据恢复等,后续文章会一一介绍。

    硬盘信息查询结果