Oracle ASM Secondary Block Copy (Mirror) Corrupted

by

Oracle ASM 如果是normal或high redundancy会有多个副本数据，ASM 始终使用Primary（主） AU 读取数据。如果Primary AU 损坏，ASM 将读取Secondary（辅助）AU。如果Secondary AU 正常，ASM 将尝试使用辅助 AU 覆盖损坏的primary AU。如果损坏的主 AU 被修复，则该 AU 将恢复为主 AU。如果无法覆盖损坏的主 AU，ASM 将尝试把新的 AU 写入磁盘上的其他位置。如果写入操作成功，则该 AU 将成为新的主 AU。但是如果损坏的是Secondary Block呢?

验证主、辅助AU

SQL> select PXN_KFFXP, -- physical extent number \
      XNUM_KFFXP, -- virtual extent number
      DISK_KFFXP, -- disk number
      AU_KFFXP,    -- allocation unit number
      decode(LXN_KFFXP,0,'Primary',1,'Secondary','header metadata') "AU type"
    from X$KFFXP
    where NUMBER_KFFXP=256 -- ASM file 272
    AND GROUP_KFFXP=4 -- group number 1
    order by 1;

SK_KFFXP   AU_KFFXP AU type
---------- ---------- ---------- ---------- ---------------
     0        0           0    144 Primary
     1        0           1    144 Secondary
     2        1           1    145 Primary
     3        1           0    145 Secondary
SQL> select PXN_KFFXP, -- physical extent number \
      XNUM_KFFXP, -- virtual extent number
      DISK_KFFXP, -- disk number
      AU_KFFXP,    -- allocation unit number
      decode(LXN_KFFXP,0,'Primary',1,'Secondary','header metadata') "AU type"
    from X$KFFXP
    where NUMBER_KFFXP=256 -- ASM file 272
    AND GROUP_KFFXP=4 -- group number 1
    order by 1;

SK_KFFXP   AU_KFFXP AU type
---------- ---------- ---------- ---------- ---------------
     0        0           0    144 Primary
     1        0           1    144 Secondary
     2        1           1    145 Primary
     3        1           0    145 Secondary

-- or --

select file_number,name from v$asm_alias where name like '%TESTBS%';
FILE_NUMBER NAME
----------- --------------------
        266 TESTBS.266.121345302

set linesize 140 pagesize 1400
col "FILE_NAME" format a30
set head on
select NAME         "FILE_NAME",
       NUMBER_KFFXP "FILE_NUMBER",
       XNUM_KFFXP   "EXTENT_NUMBER",
           LXN_KFFXP,PXN_KFFXP ,
       DISK_KFFXP   "DISK_NUMBER",
       AU_KFFXP     "AU_NUMBER",
       SIZE_KFFXP   "NUMBER_of_AUs"
  from x$kffxp, v$asm_alias
where GROUP_KFFXP = GROUP_NUMBER
   and NUMBER_KFFXP = FILE_NUMBER
   and system_created = 'Y'
   and NUMBER_KFFXP=266  and GROUP_KFFXP = 2
order by XNUM_KFFXP,LXN_KFFXP;

FILE_NAME                      FILE_NUMBER EXTENT_NUMBER  LXN_KFFXP  PXN_KFFXP DISK_NUMBER  AU_NUMBER NUMBER_of_AUs
------------------------------ ----------- ------------- ---------- ---------- ----------- ---------- -------------
TESTBS.266.1213453021                  266             0          0          0           1         21             1
TESTBS.266.1213453021                  266             0          1          1           3         25             1
TESTBS.266.1213453021                  266             1          0          2           3         26             1
TESTBS.266.1213453021                  266             1          1          3           1         22             1
...
TESTBS.266.1213453021                  266            12          0         24           1         27             1
TESTBS.266.1213453021                  266            12          1         25           2         32             1
...

ASM文件的空间分配是以ASM extent为单位，每一个extent是由一个或多个AU组成，在11.2版本，前20000个extent，每一个extent由1个AU组成，接下来的20000个extent，每一个由4个AU组成，再超出的extent，每一个由16个AU组成。这个特性被叫做可变extent。而在11.1版本，extent的增长则遵循的是1-8-64倍AU的方式。在版本10，可变extent这个特性还没出现，因此所有的extent的大小都是1个AU。

查找table block 与ASM 磁盘的映射

select file_id,block_id,blocks from dba_extents where segment_name='OBJECT' and owner='TEST' order by block_id;

   FILE_ID   BLOCK_ID     BLOCKS
---------- ---------- ----------
         ...
         6       6016        128
         6       6144        128
         6       6272        128
...
或是使用rowid生成

**# 查找datafile 对应的asm file 266,   6396 block 对应磁盘的 ASM extent_number, block number
公式 
**ASM extent=  block_number*block_size/au_size=6396*8/4096=12.4921875 取整12
**block number= 0.4921875*4,194,304=2,064,384; 2,064,384/8192=252 
**即第 12 个 extent_number, 第 252 个block**

select NAME         "FILE_NAME",
       NUMBER_KFFXP "FILE_NUMBER",
       XNUM_KFFXP   "EXTENT_NUMBER",
           LXN_KFFXP,PXN_KFFXP ,
       DISK_KFFXP   "DISK_NUMBER",
       AU_KFFXP     "AU_NUMBER",
       SIZE_KFFXP   "NUMBER_of_AUs"
  from x$kffxp, v$asm_alias
where GROUP_KFFXP = GROUP_NUMBER
   and NUMBER_KFFXP = FILE_NUMBER
   and system_created = 'Y'
   and NUMBER_KFFXP=266  and GROUP_KFFXP = 2 and XNUM_KFFXP  =12
order by XNUM_KFFXP,LXN_KFFXP;

FILE_NAME                      FILE_NUMBER EXTENT_NUMBER  LXN_KFFXP  PXN_KFFXP DISK_NUMBER  AU_NUMBER NUMBER_of_AUs
------------------------------ ----------- ------------- ---------- ---------- ----------- ---------- -------------
TESTBS.266.1213453021                  266            12          0         24           1         27             1
TESTBS.266.1213453021                  266            12          1         25           2         32             1

primary block 在 /dev/sdg au 27
mirror block  在 /dev/sdh au 32

--读取 primary                                                                        --读取 mirror
dd if=/dev/sdg bs=4M skip=27 count=1|dd bs=8k \         dd if=/dev/sdh bs=4M skip=32 count=1|dd bs=8k \    
skip=252 count=1 | dd bs=8k skip=0 count=1|od -v -x     skip=252 count=1 | dd bs=8k skip=0 count=1|od -v -x

破坏
#  dd if=/dev/zero of=/dev/xxx  bs=8k  count=1 seek=xxx

破坏primary block可以从 Secondary Block自动恢复， 但是破坏Secondary Block，因为dbv或rman 还是DB 层SQL查询，默认读的是primary block，所以读不到Secondary Block，所以只是Secondary Block坏是无感知的。

从mirror修复的日志提示

Hex dump of (file 7, block 133) in trace file /u01/app/oracle/diag/rdbms/grac4/grac41/trace/grac41_ora_29037.trc
Corrupt block relative dba: 0x01c00085 (file 7, block 133)
Completely zero block found during multiblock buffer read
Reading datafile '+TEST/grac4/datafile/test_ts.256.852905863' for corruption at rdba: 0x01c00085 (file 7, block 133)
Read datafile mirror 'TEST_0001' (file 7, block 133) found same corrupt data (no logical check)
Read datafile mirror 'TEST_0000' (file 7, block 133) found valid data
Hex dump of (file 7, block 133) in trace file /u01/app/oracle/diag/rdbms/grac4/grac41/trace/grac41_ora_29037.trc
Repaired corruption at (file 7, block 133)

解决方案

方法1 配置从哪个failgroup ，参数asm_preferred_read_failure_groups （本应用于extented RAC），后sql查询发现坏块就会自动修复。

方法2，drop disk 做reblance，自动修复

方法3，使用scrub 一个检查逻辑数据损坏并自动修复的过程

alter diskgroup xx scrub [repair] [power high|low]

e.g.

– 数据文件级别：
SQL> alter diskgroup data scrub file ‘+data/orcl/datafile/example.266.806582193’ repair power high force;
– 特定磁盘级别：
SQL> alter diskgroup data scrub disk data_0005 repair power high force;
– ASM 磁盘组级别：
SQL> alter diskgroup data scrub power low;

references

https://www.hhutzler.de/blog/how-asm-fixes-block-corruption-with-normal-redundancy-dg

https://support.enmotech.com/article/publish/12441