Sunday, March 14, 2021

Microsoft Windows update ETL Logs are continuously getting generated on Microsoft Windows Server 2016 and Microsoft Windows 10, V1607

Microsoft Windows update ETL Logs are continuously getting generated on Microsoft Windows Server 2016 and Microsoft Windows 10, V1607.

Symptoms:

The Servers will be writing new ETL logs under C:\Windows\Logs\Windows Update folder until the server runs out of disk space and eventually system goes to hung state. This will continue to happen even after setting windows update Service to manual. Restarting Windows update service will temporarily cleanup all ETL logs.

Windows update ETL logs












Cause:

     Windows Update Service makes unnecessary http calls to fe3.delivery.mp.microsoft.com or fe2.update.microsoft.com if proxy returns HTTP 407. The issue occurs after installing Windows Updates released after 2021 1B and prior to 2021 2C and 3B timeframe.

 Solution:

          Microsoft has released a cumulative update KB5000803 which has fix for this issue. Size of the package is around 1.7 GB, OS drive should have minimum 7 GB free space before attempting the installation.

Workaround:

 Block access to Windows Update Servers by policy “Turn off access to all Windows Update features”.

Steps are below;

1. Run gpedit.msc

2. Expand “Computer Configuration”

3. Expand “Administrative Templates”

4. Expand “System”

5. Expand “Internet Communication Management”

6. Click on “Internet Communication settings”

7. On the right hand side locate the setting “Turn off access to all Windows Update               features” and double click to open the settings

8. Change the setting to “Enabled” and click apply.

9. Restart Windows update Service. Perform a clean reboot if possible.



   







Blocking https://fe3.delivery.mp.microsoft.com URL on the proxy server will also prevent the issue, as the proxy will return HTTP 403 instead of HTTP 407 which causes the issue.

  Disabling authentication on the proxy server is another option to mitigate this issue.


Wednesday, December 2, 2020

SAN LUNs/Disks are not detecting by OS post SAN storage Power outage.

SAN LUNs/Disks are not detecting by OS post SAN storage Power outage.

 

Recently one of the Customer experienced SAN power outage. Many Servers with make HP, IBM, CISCO and various OS flavors were impacted. Post SAN Storage (Fiber Channel) recovery SAN disks were automatically discovered after the Server reboot.

 

However, for many IBM Servers mainly BladeCenter HS22 irrespective of OS, SAN drives did not reconnect after the Server reboot as well.

We had to perform below Steps to reconnect the SAN storage.

  1. Login to IBM bladecenter Advanced management Module
  2. Navigate to I/O Module Task
  3. SelectàAdmin/Power/Restart
  4. SelectàRestart Module(s) and Run standard Diagnostics
  5. ClickàPerform action 
  6. Reboot/Start the Server.












Excel/Office files are taking very long time to open; Virtual memory/Page file configuration is not saving on Microsoft Windows Server 2016

This blog post is to share few issues with Microsoft ‘System Monitor’ (Sysmon).


Sysmon is a Windows system service  that helps to log system activity to the Windows event log. Many customers are relaying on this tool to collect additional logging, which helps to identify malicious or anomalous activity on their environment.

Below are the two symptoms identified with Sysmon driver version 11 on Microsoft Windows Server 2016.

1.Excel/Office files are taking very long time to open/SMB share is too slow.


If anyone try to access files from a Server installed with Sysmon Version11, the client may experience poor performance. For example, when they try to open an Excel file from another Machine may take more than a minutes and it gets stuck at 0% opening excel window. Pic below;










2.Virtual memory/Page file configuration is not saving.

It create a random paging file, not as per the desired configuration, what we set for ‘Initial and Maximum’ size at ‘System properties-Advanced-Performance Option’. Suppose we configure 2048 Mb, the System page file would appear a lower value instead of configured 2048Mb post server reboot.

As part of troubleshooting, Sysmon version11.0 was removed and reinstalled with Sysmon Version12.0, which resolved both the above issues.


[Solved] HP ILO, iBMC Remote console and Virtual Media Applets are not working.

HP ILO, iBMC Remote console and Virtual Media Applets are not working.

            Remote Server management using Integrated Lights-out Management Module (ILO) for HP Servers and Integrated Baseboard Management Controller (iBMC) for IBM servers have many challenges due to the enhanced security features of Java JVM, Web browsers and Operating System hardening. Most of the case the Remote Console (Java applet-based console) will not open with many errors.

 

This post will help you with a workaround to access the ILO/iBMC console using Integrated Remote Console or Java JVM web console.

Common Issues when connecting to Server Console form ILO/iBMC.

1. Microsoft Internet Explorer is required to Run Integrated Remote Console. When connecting Remote Console using Integrated Remote Console.










Follow below steps to resolve above described issue.

On Internet Explorer Click on Gear/Tool iconCompatibility View Settings Add ILO/iBMC IP.

Restart Internet Explorer and Try again.










2.Remote Console KVM from a Java Applet based is throwing ClassNotFoundExepetion Error.















Follow below steps to resolve this error.

Click Windows Start MenuControl PanelClick on Java icon from ‘All control Panel Items’

Select SecurityEdit Site List Add ‘ILO/iBMC IP address followed by *.

And add ‘ILO/iBMC IP’ address followed by :443











Next Click on ‘Java Control Panel’ Advanced Menu, Scroll down and deselect TLS1.1, TLS1.2, and TLS1.3

Enable TLS 1.0 Only 









Re-open Internet Explorer browser and follow below steps to access ILO Console.

Login to ILO Remote ConsoleSelect Remote Console KVM Java Applet BasedClick on Continue for Security related Warnings 








Select “I accept the risk and want to Run this Application”












Finally!!! You will have access to Java JVM remote console. The Virtual Media Applet also would connect without any issues.