Oracle Enterprise Manager provides you with the ability to monitor your
environments and alert you once specified thresholds have been reached.
Blackouts allow you to suspend monitoring so you do not get notified.
This is useful for scheduled maintenance windows, such as cold backups,
where the application and/or database may not be available.
As well, blackouts also suspends data collection for the given targets.
This means that certain metrics such as availability will not be
affected.
To create a blackout from the GUI, login to Enterprise Manager, navigate
to the target you would like to blackout and at the bottom of the page
under Related Links you will see a
Blackouts link. You will be brought to the following page:
On the screen above you can view any blackouts that may currently be in
effect as well as create new ones. To create a new blackout click on
the
create button.
![clip_image002[8]](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj-5jq48UripzgoW0Q58yFLh2FjpZD-d_qHG6NxT560moY08yCqSOaD3waGDJDIL5QNMUztlvz47uA7wR0W7Fu9lPR8lkRBT6yTmRGiHjqAqN6wkUMQ8WSbudJUcgRO2nPrDI5ybx3GJrQ/?imgmax=800 "clip_image002[8]")
On this page you can create a name for the blackout, with the default
being “Blackout-<timestamp>”. You can also select the targets you
wish to set the blackout for. As you can see from the screenshot I am
going to set a blackout for an infrastructure application server
(infra10g).
You can also provide a reason for the blackout by click on the
Reason drop down list. Quite a few are available, from
Server Bounce to
Security Patch. Jobs can be disabled by deselecting the
Run jobs during the blackout
checkbox. If your applying a security patch then you may not want a
scheduled backup to run as it will either error or cause problems.
![clip_image002[10]](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjlX065jK1ziJxAgmJDU0MNIdQqZPXXiOg5-pZ38bRc3knten8TuSrSkR6N2uaRy-uBsSJgTBzf2JnVi-PgCrD3iDxkUkvXrGOcsVFSKdUKzUJIqOWTxR02Zx45EIJRPS-w7uxgsmsThmU/?imgmax=800 "clip_image002[10]")
The next screen allows you to select which components within the target
will be blacked out. I selected a full blackout but you can select
certain members if the outage will only affect specific components.
![clip_image002[12]](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhqhXmyr4CW4PkwTsAUzc7YHiJwIp5o3W_RyIjtG6sZkO94OqnpQL0I525pNOh6vHDU5ZHIQjimL6W6R_id6SVuMjuEkdYQRfsbpSZ_-khQhQZWv_DBhiHfYvG6A7f5DSGWRbB5i-jkefw/?imgmax=800 "clip_image002[12]")
This screen allows you to schedule the blackout. It can either start
immediately or you can choose a date along with a duration. Blackouts
can be repeating as well, so you only have to create one for that
monthly maintenance window for example.
![clip_image002[14]](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiN9OviFh9SGEJCoAjJiuiZXg1jm-ToEOTCY-N_fpCe4-GEwZZT-96QzvVcPflFgElPM27t9w5ZE0j8J6lxlE041OzWO4yoTiEBM2LoSqMDQP79JaMzI0YPo5nsT-Jk368Pp7KgZqslgKg/?imgmax=800 "clip_image002[14]")
The last screen provides a summary and once you have finished reviewing click on the
Finish button.
You can also set blackouts from the command line, which is useful if you
have some maintenance scripts which are not executed from Enterprise
Managers job system. I’ve only tested this on linux but it should be
the same for windows.
To set a blackout for a list of targets:
emctl start blackout <Blackoutname> [<Target_name>[:<Target_Type>]]…. [-d Duration]
To set a blackout for all targets on a host:
emctl start blackout <Blackoutname> [-nodeLevel] [-d <Duration>]
-nodeLevel tells the agent to stop monitoring all targets on the server.
-d Duration allows you to set a duration in the format of [days]
hh:mm. ex. 1 02:05 means the blackout will last for 1 day, 2 hours and
5 minutes.
For example, to use this in a script in which all targets will be
unavailable you would start a blackout at the beginning of the script
and stop it at the end:
cd $AGENT_HOME/bin
./emctl start blackout alltargets-myserver –nodeLevel
<Perform Maintenance Tasks>
cd $AGENT_HOME/bin
./emctl stop blackout alltargets-myserver
Troubleshooting
In case you hit issues with blackouts take a look at the following notes:
Subject: Agent Blackout Initiated By Emctl Command Not Ending Doc ID: 559577.1
Subject: EMDiagkit Download and Master Index Doc ID: 421053.1
Subject: How to Troubleshoot the EM 10gR1 Blackout Sub-system Doc ID: 284024.1
Subject: Troubleshooting Blackouts in EM 10g Grid Control using EMDiag Kit Doc ID: 300671.1
They provide alot of information and solutions to different scenarios.
I hit an issue over the weekend in which the blackout didn’t end
properly. When I tried to stop it from the command line:
[oracle@myserver ~]$ /u01/app/oracle/product/agent10g/bin/emctl stop blackout alltargets-myserver
Oracle Enterprise Manager 10g Release 3 Grid Control 10.2.0.3.0.
Copyright (c) 1996, 2007 Oracle Corporation. All rights reserved.
Blackout stop Error : Blackout name alltargets-myserver is invalid
When trying to end the blackout via Enterprise Manager:
Error
stopping the blackout on "infra10g": ORA-20710: Agent-side blackouts
cannot be edited or stopped ORA-06512: at "SYSMAN.MGMT_BLACKOUT_ENGINE",
line 501 ORA-06512: at "SYSMAN.MGMT_BLACKOUT_ENGINE", line 3262
ORA-06512: at "SYSMAN.MGMT_BLACKOUT", line 74 ORA-06512: at
"SYSMAN.MGMT_BLACKOUT_UI", line 1167 ORA-06512: at line 1 .
To fix this problem I performed the following:
1. Shutdown the agent on the target server
myserver
2. Removed the blackouts.xml file under $AGENT_HOME/sysman/emd
3. Used note 421053.1 to install the EMDiag kit
4. Logged in as sysman on the Enterprise Managers repository database and executed the following query:
select blackout_guid, blackout_name
from mgmt_blackouts;
BLACKOUT_GUID BLACKOUT_NAME
-------------------------------- ----------------------
30E2956CA329F0E59FBDF50951F2578E alltargets-myserver
5. Then executed:
exec mgmt_diag.KillBlackout(HEXTORAW(‘30E2956CA329F0E59FBDF50951F2578E’));
6. Restarted the agent on
myserver and when I looked in Enterprise manager the blackout had cleared.
I have seen the command used above for some other scenarios but not this
one specifically. Before executing any commands in your environment
please test first.
![clip_image002[8]](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjGtDEsviVmnHkyysW0O2g_neTJ8fZNNeWXAg_8nW6M346gbZJggrELL92rdb48H_5oQGmuN7NO6yBceu9eal1PBnlgKz8FmTQQcf3hol-P9yPlDEcX8QpcVdk2PLqvw-r66qmmw9IcT9Y/s1600-h/clip_image002%5B8%5D%5B2%5D.jpg "clip_image002[8]")
![clip_image002[10]](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiwbz6ONdvaf89WDQEKQhY8MI_5cuNTs2_HYYYetRwEf9atREAqlmW_lUnbJcqnp6nO24aJSsNVEbgMKu-CfnEnH6OIWBLJhQtV9zSrlLVS2EqjdxW-QEtqeb9vzvPCbxgilKO7GhW0eB0/s1600-h/clip_image002%5B10%5D%5B2%5D.jpg "clip_image002[10]")
![clip_image002[12]](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgHa3Plkvx3w5B8dZnVQCgitLLHCYCvX1hH4hLoxamDyZzP5oo6-nA60cpKi1tnTdZL7_Qv_PXNgleDrdYwft9nbqQE2I_ufNTjpu9I6aqT1FwHBW4JX8VD_eTH8xNqQBa8mkHy3ZcC6JE/s1600-h/clip_image002%5B12%5D%5B2%5D.jpg "clip_image002[12]")
![clip_image002[14]](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhGSn0zjWqyWOyrQiol8crrbSEKvaiksolaVMyUMS-cBaFOi7MHzUWmWDTGdriL34aJmz2Yaxf0kxWvCTsS-r0DfGof5g3F1rLQRFeqXlc3U0ruRkr7rDs9CL2EcBSdkh63l5JlUhVsIY8/s1600-h/clip_image002%5B14%5D%5B2%5D.jpg "clip_image002[14]")