Oracle19c使用USE_LARGE_PAGES可在LINUX平台的自动配置hugepage
Hugepage是linux平台oracle数据库的建议配置，同样PostgreSQL等其它使用共享内存和多进程的系统都建议使用hugepage, 默认的4K配置带来的pagetables内存空间非常大。通常是修改LINUX内核参数sysctl.conf配置中hugepage大小和页个数，在oracle没有使用AMM时配置使用hugepage.在oracle数据库参数中与大页相关的参数为USE_LARGE_PAGES。在19c中可以使用auto_only值可以在OS未预先配置hugepage的情况下，oracle db实例启动时自动按需扩展linux kernel中分配hugepage.
USE_LARGE_PAGES参数引入从11.2.0.2, 当时的参数有:true(default),false,only.
true分别表示使用大页，但是预分配的hugepages不够用oracle实例在启动时完全不用，如果够用再使用；在11.2.0.3中，该行为已更改，以使Oracle现在可以在大页面中分配SGA的能力，如果用完了，它将使用小页面分配SGA的其余部分混合页；
false表示不使用大页。
only表示只使用大页，如果操作系统分配大页不足，数据库实例在启动时会失败。
同时在11.2.0.3 还引入了一个可能的 possible value，AUTO，是undocument parameter,同时也是不被oracle服务支持的。该参数在12.2可以使用，用于在linux系统中oracle自动的配置hugepage, 在19c It was desupported，虽然V$PARAMETER_VALID_VALUES中还保留，但是会提示ORA-27107
[oracle@oel7db1 ~]$ oerr ora 27107 27107, 0000, "AUTO value for USE_LARGE_PAGES parameter is no longer supported" // *Cause: The USE_LARGE_PAGES configuration parameter was set to AUTO. // *Action: Consult the alert file for details.
从19c开始引入了另一个值AUTO_ONLY, 看来Auto还是有些不知的原因被抛弃。 auto_only是auto+only的综合，表示自动扩展使用hugepage,并且只使用，如果无法分配hugepages，实例在启动时则会失败。在EXADATA中AUTO_ONLY为默认值，on-premise版本中true仍是默认值，不确定AUTO_ONLY的默认设置是否仅适用于Exadata，例如在我的Oracle 19.3非Exadata的VM中，默认值为“ TRUE”（意味着Oracle可以使用它获取的任何大页面+任何剩余内存作为小页面分配）。AUTO_ONLY的主要优点在于，我们不再需要重新计算设置操作系统所用的大页面总数，可以让数据库本身从可用内存中创建大页面，从而在大型系统上进行配置，尤其是进行快速编排 容易得多
AUTO_ONLY
This setting is available starting with Oracle Database 19c and it is the default setting for Exadata systems. It specifies that, during startup, the instance will calculate and request the number of large pages it requires. If the operating system can fulfill this request, then the instance will start successfully. If the operating system cannot fulfill this request, then the instance will fail to start. This ensures that no instances will run with under-provisioned large pages.
Note:USE_LARGE_PAGES is set to FALSE automatically in an Oracle ASM instance when MEMORY_TARGET is enabled. In this case, the FALSE setting does not cause performance degradation.
下面开始测试, Oracle linux7 + Oracle 19.3 on-premise，
1, 不使用大页
[root@oel7db1 ~]# cat /proc/meminfo MemTotal: 3765384 kB MemFree: 3557852 kB MemAvailable: 3623672 kB Buffers: 2108 kB Cached: 107400 kB SwapCached: 0 kB ... Shmem: 8704 kB Slab: 28048 kB SReclaimable: 15932 kB SUnreclaim: 12116 kB KernelStack: 2128 kB PageTables: 4432 kB ... CmaFree: 0 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB DirectMap4k: 65472 kB DirectMap2M: 4128768 kB SQL> startup mount ORACLE instance started. Total System Global Area 1073738888 bytes Fixed Size 9143432 bytes Variable Size 792723456 bytes Database Buffers 268435456 bytes Redo Buffers 3436544 bytes Database mounted. SQL> show parameter large PARAMETER_NAME TYPE VALUE ------------------------------------------------------------ ----------- ---------------------------------------------------------------------------------------------------- large_pool_size big integer 0 use_large_pages string FALSE # db alert log **************************************************** /dev/shm will be used for creating SGA Large pages will not be used. Only standard 4K pages will be used **************************************************** ********************************************************************** Dump of system resources acquired for SHARED GLOBAL AREA (SGA) Per process system memlock (soft) limit = 128G Expected per process system memlock (soft) limit to lock instance MAX SHARED GLOBAL AREA (SGA) into memory: 1024M Available system pagesizes: 4K, 2048K Supported system pagesize(s): PAGESIZE AVAILABLE_PAGES EXPECTED_PAGES ALLOCATED_PAGES ERROR(s) 4K Configured 262150 262150 NONE Reason for not supporting certain system pagesizes: 2048K - Dynamic allocate and free memory regions **********************************************************************
2, AUTO_ONLY 但是AMM
SQL> @pvalid use_large_pages
Display valid values for multioption parameters matching "use_large_pages"...
PAR# PARAMETER ORD VALUE DEFAULT
------ -------------------------------------------------- ---------- ------------------------------ -------
167 use_large_pages 1 TRUE DEFAULT
use_large_pages 2 AUTO
use_large_pages 3 ONLY
use_large_pages 4 FALSE
use_large_pages 5 AUTO_ONLY
SQL> alter system set use_large_pages=AUTO_ONLY scope=spfile;
System altered.
SQL> shut abort
ORACLE instance shut down.
SQL> startup mount
ORA-27138: unable to allocate large pages with current parameter setting
Additional information: 9571
# ALERT LOG
Starting ORACLE instance (normal) (OS id: 2911)
****************************************************
/dev/shm will be used for creating SGA
Large pages will not be used. Only standard 4K pages will be used
****************************************************
**********************************************************************
Dump of system resources acquired for SHARED GLOBAL AREA (SGA)
Per process system memlock (soft) limit = 128G
Expected per process system memlock (soft) limit to lock
instance MAX SHARED GLOBAL AREA (SGA) into memory: 1024M
Available system pagesizes:
4K, 2048K
Supported system pagesize(s):
PAGESIZE AVAILABLE_PAGES EXPECTED_PAGES ALLOCATED_PAGES ERROR(s)
4K Configured 262150 262150 NONE
Reason for not supporting certain system pagesizes:
2048K - Dynamic allocate and free memory regions
**********************************************************************
3， 禁用AMM, auto_only
--使用pfile恢复上面参数use_large_pages为true SQL> show parameter target PARAMETER_NAME TYPE VALUE ------------------------------------------------------------ ----------- ---------------------------------------------------------------------------------------------------- memory_max_target big integer 1G memory_target big integer 1G parallel_servers_target integer 20 pga_aggregate_target big integer 0 sga_target big integer 0 target_pdbs integer 2 SQL> alter system set memory_target=0; System altered. SQL> show parameter sga PARAMETER_NAME TYPE VALUE ------------------------------------------------------------ ----------- -------------------------------------- allow_group_access_to_sga boolean FALSE lock_sga boolean FALSE pre_page_sga boolean TRUE sga_max_size big integer 668M sga_min_size big integer 0 sga_target big integer 668M unified_audit_sga_queue_size integer 1048576 SQL> show parameter target PARAMETER_NAME TYPE VALUE ------------------------------------------------------------ ----------- -------------------------------------- memory_max_target big integer 0 memory_target big integer 0 parallel_servers_target integer 20 pga_aggregate_target big integer 356M sga_target big integer 668M target_pdbs integer 1 SQL> show parameter large PARAMETER_NAME TYPE VALUE ------------------------------------------------------------ ----------- ---------------------------------------------------------------------------------------------------- large_pool_size big integer 0 use_large_pages string TRUE SQL> alter system set use_large_pages=AUTO_ONLY scope=spfile; System altered. SQL> shut immediate; ORA-01109: database not open Database dismounted. ORACLE instance shut down. SQL> startup mount ORACLE instance started. Total System Global Area 700445640 bytes Fixed Size 9139144 bytes Variable Size 419430400 bytes Database Buffers 268435456 bytes Redo Buffers 3440640 bytes Database mounted. SQL> # db alert log 2020-11-04 05:04:22.275000 -05:00 Starting ORACLE instance (normal) (OS id: 3567) **************************************************** Sys-V shared memory will be used for creating SGA **************************************************** DISM started, OS id=3579 ********************************************************************** Dump of system resources acquired for SHARED GLOBAL AREA (SGA) Per process system memlock (soft) limit = 128G Expected per process system memlock (soft) limit to lock instance MAX SHARED GLOBAL AREA (SGA) into memory: 672M Available system pagesizes: 4K, 2048K Supported system pagesize(s): PAGESIZE AVAILABLE_PAGES EXPECTED_PAGES ALLOCATED_PAGES ERROR(s) 2048K 336 336 336 NONE Reason for not supporting certain system pagesizes: 4K - Large pagesizes only **********************************************************************
Note:
已自动使用上了HUGEPAGES, 并且大小刚好也是我们配置的sga大小，没有太多的浪费。
查看操作系统
$cat /proc/meminfo MemTotal: 3765384 kB MemFree: 2246868 kB MemAvailable: 2721796 kB Buffers: 2108 kB Cached: 515388 kB Shmem: 8800 kB HardwareCorrupted: 0 kB AnonHugePages: 0 kB ShmemHugePages: 0 kB ShmemPmdMapped: 0 kB CmaTotal: 0 kB CmaFree: 0 kB HugePages_Total: 336 HugePages_Free: 3 HugePages_Rsvd: 3 HugePages_Surp: 0 Hugepagesize: 2048 kB DirectMap4k: 67520 kB DirectMap2M: 4126720 kB $sysctl -a|grep page vm.hugepages_treat_as_movable = 0 vm.nr_hugepages = 336 vm.nr_hugepages_mempolicy = 336 vm.nr_overcommit_hugepages = 0 vm.page-cluster = 3 [oracle@oel7db1 dbs]$ cat /etc/sysctl.conf |grep page [oracle@oel7db1 dbs]$
Note:
可见已修改了vm.nr_hugepages，配置了hugepage,但是并未改变 /etc/sysctl.conf的配置，重启操作系统hugepage将会释放或只按/etc/sysctl.conf指定的大小。那是哪个进程分配了内存呢？从上面的db alert log中可以看到OS PID 3579。
[oracle@oel7db1 ~]$ ps -ef|grep 3579|grep -v grep root 3579 1 0 08:19 ? 00:00:00 ora_dism_anbob19c [oracle@oel7db1 ~]$ ls -l `which oradism` -rwsr-x--- 1 root oinstall 147848 Apr 17 2019 /u01/app/oracle/product/19.2.0/db_1/bin/oradism
Note:
实际是有oradism process扩展了linux kenel 的hugepage个数。该进程是root的属主。另外如果启动时linux 预分配的hugepages够实例使用，实例启动时也不会调用oradism，alert中不会看到该进程，该进程也不会随实例启动。
另外发现在当前的环境中使用strace 跟踪数据库启动过程，AUTO_ONLY是无法启动的。
[oracle@oel7db1 ~]$ strace -f -o output.txt sqlplus / as sysdba SQL*Plus: Release 19.0.0.0.0 - Production on Wed Nov 4 08:19:02 2020 Version 19.3.0.0.0 Copyright (c) 1982, 2019, Oracle. All rights reserved. Connected to an idle instance. SQL> startup ORA-27106: system pages not available to allocate memory Additional information: 6122 Additional information: 2 Additional information: 3 SQL> exit # alert log Starting ORACLE instance (normal) (OS id: 1928) **************************************************** Sys-V shared memory will be used for creating SGA **************************************************** WARNING: ------------------------------- WARNING: oradism did not start up correctly. Return code: 16 errno 0 info1 54321 info2 65535 ---------------------------------------- Oradism binary does not have root privilege. Please verify if oradism has required privilege Oradism spawned failed for large page allocation ERROR: Failed to get available system pages to allocate memory ********************************************************************** Dump of system resources acquired for SHARED GLOBAL AREA (SGA) Per process system memlock (soft) limit = 128G Expected per process system memlock (soft) limit to lock instance MAX SHARED GLOBAL AREA (SGA) into memory: 672M Available system pagesizes: 4K, 2048K Supported system pagesize(s): PAGESIZE AVAILABLE_PAGES EXPECTED_PAGES ALLOCATED_PAGES ERROR(s) 2048K 0 336 0 ORA-27102 Reason for not supporting certain system pagesizes: 4K - Large pagesizes only RECOMMENDATION: 1. Configure system with expected number of pages for every supported system pagesize prior to the next instance restart operation. ********************************************************************** SGA: Realm creation failed [oracle@oel7db1 ~]$ oerr ora 27106 27106, 00000, "system pages not available to allocate memory" // *Cause: System page count for supported page sizes was misconfigured. // *Action: Configure system page count as recommended in the alert file. [oracle@oel7db1 ~]$ oerr ora 27102 27102, 00000, "out of memory" // *Cause: Out of memory // *Action: Consult the trace file for details
— over —
微信扫一扫，打赏作者吧～
对不起，这篇文章暂时关闭评论。