2016-1-8 ORA-1652: unable to extend temp segment by 128 in tablespace解决方案
一次ORA-1652的诊断过程,系统不能使用,重启后可以使用。
weblogic日志:
####<2016-1-8 上午09时40分27秒 CST>
“, which is more than the configured time (StuckThreadMaxTime) of “600” seconds in “server-failure-trigger”. Stack trace:
java.net.SocketInputStream.socketRead0(Native Method)
java.net.SocketInputStream.read(SocketInputStream.java:152)
java.net.SocketInputStream.read(SocketInputStream.java:122)
oracle.net.ns.Packet.receive(Packet.java:300)
oracle.net.ns.DataPacket.receive(DataPacket.java:106)
####<2016-1-8 上午09时56分53秒 CST>
GET /web/menu_icon/zongheguanli.png?tSession=1452217537514 HTTP/1.1
X-Forwarded-For: 10.119.124.109
Accept: */*
Referer: http://10.121.180.109/web/app/MyApp.jsp
Accept-Language: zh-CN
User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; Trident/5.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E; GWX:DOWNLOADED)
Accept-Encoding: gzip, deflate
Connection: Keep-Alive
Cookie: DWRSESSIONID=UmKxc9wQCOJBrOHXa$Zx1oNVu9l; JSESSIONIDMINI=BtcLWPJprp2xpBhqbsgv7NzKmJ8yLZhvxFLpThyGcz50JzGyh4vH!2102636630; ComtopSessionSID=
]“, which is more than the configured time (StuckThreadMaxTime) of “600” seconds. Stack trace:
weblogic.common.resourcepool.ResourcePoolImpl.reserveResourceInternal(ResourcePoolImpl.java:385)
weblogic.common.resourcepool.ResourcePoolImpl.reserveResource(ResourcePoolImpl.java:342)
weblogic.common.resourcepool.ResourcePoolImpl.reserveResource(ResourcePoolImpl.java:329)
weblogic.jdbc.common.internal.ConnectionPool.reserve(ConnectionPool.java:417)
weblogic.jdbc.common.internal.ConnectionPool.reserve(ConnectionPool.java:324)
alert日志:
Fri Jan 08 09:41:39 2016
ORA-1652: unable to extend temp segment by 128 in tablespace TEMP
Fri Jan 08 09:41:39 2016
ORA-1652: unable to extend temp segment by 128 in tablespace TEMP
Fri Jan 08 10:01:36 2016
ORA-1652: unable to extend temp segment by 128 in tablespace TEMP
Fri Jan 08 10:11:35 2016
ORA-1652: unable to extend temp segment by 128 in tablespace TEMP
AWR:
Snap Id | Snap Time | Sessions | Cursors/Session | Instances | |
---|---|---|---|---|---|
Begin Snap: | 5732 | 08-Jan-16 09:00:48 | 548 | 73.3 | 2 |
End Snap: | 5733 | 08-Jan-16 10:00:59 | 545 | 73.0 | 2 |
Elapsed: | 60.18 (mins) | ||||
DB Time: | 126.97 (mins) |
Event | Waits | Total Wait Time (sec) | Wait Avg(ms) | % DB time | Wait Class |
---|---|---|---|---|---|
DB CPU | 6414.3 | 84.2 | |||
direct path write temp | 323,000 | 479.8 | 1 | 6.3 | User I/O |
Elapsed Time (s) | Executions | Elapsed Time per Exec (s) | %Total | %CPU | %IO | SQL Id | SQL Module | SQL Text |
---|---|---|---|---|---|---|---|---|
3,579.67 | 1,666 | 2.15 | 46.99 | 99.76 | 0.00 | ga37qxa3p485g | JDBC Thin Client | select tddakentit0_.ID as ID66… |
2,917.88 | 6 | 486.31 | 38.30 | 81.96 | 16.41 | dxcdycgfr1w5n | select nvl(round(avg(ctime), 2… | |
190.66 | 6 | 31.78 | 2.50 | 14.15 | 85.29 | 8yavyxsts8hu9 | select a.tablespace_name table… | |
28.21 | 6 | 4.70 | 0.37 | 99.74 | 0.00 | brb01q8dygvy6 | select SQL_TEXT d_maxtime_sql … | |
16.66 | 6 | 2.78 | 0.22 | 99.74 | 0.00 | 45b35s83zwmah | select round(max(ELAPSED_TIME)… | |
11.57 | 2 | 5.79 | 0.15 | 58.51 | 0.00 | 4x91zq0fkd05y | DECLARE job BINARY_INTEGER := … | |
11.57 | 6 | 1.93 | 0.15 | 58.51 | 0.00 | 4snzdp176n9wv | CALL C_DATA_SYNCH_MAI… | |
11.34 | 2 | 5.67 | 0.15 | 59.11 | 0.00 | 5k565njgcd272 | CALL G_DATA_SYNCH(‘MM… | |
7.74 | 246 | 0.03 | 0.10 | 97.86 | 0.00 | 2h1xrpzcygsdz | JDBC Thin Client | select distinct p.portlet_id, … |
6.70 | 1,729 | 0.00 | 0.09 | 87.11 | 0.00 | dfb1dmsvm3nx8 | JDBC Thin Client | select m.media_id, m.storage_p… |
检查temp表空间select tablespace_name,file_name,bytes/1024/1024 file_size,autoextensible from dba_temp_files;
发现有32G,且是自动扩展的。
select sql_id, count(*) from gv$session where event = ‘direct path write temp’
group by sql_id order by count(*) desc;
--查询无数据
select sql_id, count(*) from dba_hist_active_sess_history
where event = ‘direct path write temp’
group by sql_id order by count(*) desc;
SQL_ID COUNT(*)
-—————— —————
dxcdycgfr1w5n 71739
3v1z2vmmzadm4 16
bmndrgc7xkmjb 12
3mnjv5g06mv0x 7
看sql_id为dxcdycgfr1w5n就是AWR中那条执行慢的SQL
dxcdycgfr1w5n | select nvl(round(avg(ctime), 2), 0) d_lock_wait_time from v$lock where REQUEST > 0 |
先看下这条SQL的执行计划:
select nvl(round(avg(ctime),2),0) d_lock_wait_time from v$lock where
REQUEST > 0
Plan hash value: 2399206389
-—————————————————————————————————————
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)|
-—————————————————————————————————————
| 0 | SELECT STATEMENT | | | | 1 (100)|
| 1 | SORT AGGREGATE | | 1 | 63 | |
|* 2 | HASH JOIN | | 1 | 63 | 0 (0)|
| 3 | MERGE JOIN CARTESIAN | | 100 | 3800 | 0 (0)|
|* 4 | FIXED TABLE FULL | X$KSUSE | 1 | 19 | 0 (0)|
| 5 | BUFFER SORT | | 100 | 1900 | 0 (0)|
| 6 | FIXED TABLE FULL | X$KSQRS | 100 | 1900 | 0 (0)|
| 7 | VIEW | GV$_LOCK | 10 | 250 | 0 (0)|
| 8 | UNION-ALL | | | | |
|* 9 | FILTER | | | | |
| 10 | VIEW | GV$_LOCK1 | 2 | 50 | 0 (0)|
| 11 | UNION-ALL | | | | |
|* 12 | FIXED TABLE FULL| X$KDNSSF | 1 | 77 | 0 (0)|
|* 13 | FIXED TABLE FULL| X$KSQEQ | 1 | 77 | 0 (0)|
|* 14 | FIXED TABLE FULL | X$KTADM | 1 | 77 | 0 (0)|
|* 15 | FIXED TABLE FULL | X$KTATRFIL | 1 | 77 | 0 (0)|
|* 16 | FIXED TABLE FULL | X$KTATRFSL | 1 | 77 | 0 (0)|
|* 17 | FIXED TABLE FULL | X$KTATL | 1 | 77 | 0 (0)|
|* 18 | FIXED TABLE FULL | X$KTSTUSC | 1 | 77 | 0 (0)|
|* 19 | FIXED TABLE FULL | X$KTSTUSS | 1 | 77 | 0 (0)|
|* 20 | FIXED TABLE FULL | X$KTSTUSG | 1 | 77 | 0 (0)|
|* 21 | FIXED TABLE FULL | X$KTCXB | 1 | 77 | 0 (0)|
-—————————————————————————————————————
实际测试:
SQL> select nvl(round(avg(ctime), 2), 0) d_lock_wait_time from v$lock where REQUEST > 0;
select nvl(round(avg(ctime), 2), 0) d_lock_wait_time from v$lock where REQUEST > 0
ERROR at line 1:
ORA-01652: unable to extend tem segment by 128 in tablespace TEMP
Elapsed:00:07:50.70
SQL>exec dbms_stats.gather_fixed_objects_stats();
再次执行只需要0.3s,执行计划不是笛卡尔积了。
对于RAC的情况下,两个节点都要收集,不然,还有问题。
还没有评论,来说两句吧...