Posts Tagged ‘vCenter’

Free VMware vCenter Log Insight book on Understanding Real-time Log Analytics!

April 16th, 2014

Understanding Real-time Log Analytics

I’ve been meaning to post this blog for a few months now, but hey here it is now!

This little gem was posted some time ago Free eBook: vCenter Log Insight sweet post right?! I mean, I’m sure you’ve played with vCenter Log Insight at this point like VMware Log Insight EXPOSED! Splitting your Syslog with an axe!

But even if you haven’t check it out and get a dive into this cool technology to advance your datacenter!

Click here to get your free copy of Understanding Real-time Log Analytics. You’ll need to fill out a little form but beyond that you’re set! Check it out! <3

Tags: , , , ,
Posted in Free, Virtualization, vmware | Comments (0)

Disabling alarms in vCenter and ignoring “Health Status Monitoring” Errors in vCOPS!

January 24th, 2014

I don’t know about you, but I hate it when I login to vCOPS only to find that ALL of my vCenters show my health status as 0 and red, removing any real chance of actually seeing if there are any real PROBLEMS going on. It’s annoying as all hell and really removes the chance to actually see what may be going on.   I know what you’re saying though, I can go in to the alerts and clear that Health error.  Yea, I can do a lot of things, but if all I’m going to be doing is removing the alert and effectively ignoring it, then the error itself serves no meaning and should be removed!   6

So, here’s how you go about actually disabling this Alert from showing up in vCenter.  Disclaimer; Even with this enabled it will OCCASIONALLY still show up in vCOPS. I don’t know why yet, and when I solve as to WTF that means I’ll publish said results.   Though in the meantime here are the steps!



Yea you’ve seen this error! The sky is falling, and all that.  Now if you find this error to be pointless and stupid, here are two ways to go about clearing it up!


Firstly, you can pop yourself into the Alarm settings tab, simply uncheck the box to “Enable this Alarm” and bam, the error will no longer ‘alert’ you and appear in your alarms section.  This is awesome. But if you have more than one vCenter like me, I mean not that my 100 vCenters really affords me the need to script this… But sometimes we want to disable alerts everywhere!  

You can run this script and it’ll show you what the status of particular Alarm settings are.  As you can see here, this is configured as “True”


Get-AlarmDefinition -Entity (Get-Folder -NoRecursion) -Name "Health Status Monitoring"


Get-AlarmDefinition -Entity (Get-Folder -NoRecursion) -Name "Health Status Monitoring" | Set-AlarmDefinition -Enabled:$False

By using the Set-AlarmDefinition function of PowerCLI we can very easily change the status of this alarm from True to False, effectively disabling the alarm and setting us up for “teh win”.   What is even more awesome, is if you have other Alarms you’d like to disable, like License Logging Monitor and various other alarms you can simply run the same syntax changing the –Name and disable away!  Awesome, right?!

This personally saves me loads of time and not having to login to every vCenter to clear the stupid Alarm from appearing in the list.   Hope this helps you!

Tags: , , , , ,
Posted in PowerCLI, vCenter Operations Manager, vCOPS, vmware | Comments (1)

OMG It’s Finally here! Infinio Accelerator for NFS by @InfinioSystems goes GA!

November 5th, 2013

Well it is finally here! Are you excited? You ought to be!

Who or what IS Infinio?!?

So I know what you’re thinking, WHAT exactly is this?!  I first came to know Infinio earlier this year when I saw just how awesome this solution is.  Imagine a solution using server-side caching to minimize and off-load I/O requests so only what is required is sent back to the your shared storage.   It essentially takes 8GB of idle memory on your ESXi hosts and creates a deduplicated cache in memory.   Okay, that’s a lot of jibber-jabber, what exactly does that mean?

If you’re using NFS today you can simply download and deploy the tool in minutes.  And then all of a sudden you’re ready to rock and roll!

1.  Connect to vCenter 2.  Choose Datastore3.  Validate Hosts

4.  Preview Installation

The installation is as easy as this.  And then you’re ready to go with your NFS datastore being accelerated.

I’ve talked with the engineers and creators of this product and I was excited for this when it was in Alpha and Beta.  With it being GA this is infinitely more exciting!  The coolest part is, this isn’t simply a one-off solution which will impact your operations individually.  It scales with more installations!

Your Infinio Cache scales with your installations!

Fortunately unlike most products these days, the dashboard doesn’t look like a disgusting pile of sin.  It’s pretty and provides relevant information!

The Infinio Accelerator Dashboard actually provides relevant information!

Couple of cool things associated with this release!

  • No new hardware. No reboots. No interruptions in service to install or implement.
  • Accelerates all workload types
  • $499 per host socket – scale and grow as you need
  • Download on a 30 day trial and license when complete – The true “Try before you Buy”

This can basically turn your NFS into providing Flash-like performance accelerating whatever your backend storage happens to be.   Hey, worst case you can give it a try and see it for yourself!

For more details you can check out the Tech Field Day earlier this year where  Peter Smith Demonstrates Infinio Accelerator

In addition today with the special announcement day there will be a Live Webinar with Q/A today Nov 5 at 3PM EST.

I’ll unfortunately miss this awesome announcement webinar because… well, Afghanistan, timezones and I’ll be asleep at the 1AM or so that it’s at! :)

So check it out, be awesome, and rock rock rock!

Tags: , , , , , ,
Posted in Infinio, Infinio Accelerator, NFS, Storage, Virtualization, vmware | Comments (0)

How to detect VMFS3 and VMFS3 Upgraded Datastores with PowerCLI; Now with more sugar!

July 19th, 2013

You’re not new to Virtualization, this isn’t your first VMware rodeo, but you find yourself starting to question… OMG DO I HAVE VMFS3 DATASTORES STEATHILY HIDING IN MY SYSTEM?! I mean you do your due diligence, you check and confirm that it says VMFS5 and that it has a 1MB (Universal) block size, but yet… you’re still not sure… Hell, you might even be saying WTF? 1MB BLOCK SIZES? WTFS?!  Well, hopefully this helps break through some of the barriers to not only identify whether you have VMFS3 datastores period, whether they’re actually stealthily hiding!

What’s the big deal with upgrading VMFS3 to VMFS5?

Yea, you read the VMware documentation like this; How VMFS5 Differs from VMFS3 – Basically by reading that you come to the conclusion of ITS EASY, JUST DO IT, YAY!  But to quote Jason Corbett @NGTJason “migrate > upgrade”

Why exactly though? I mean if you read what Cormac Hogan @VMwareStorage wrote so long ago vSphere 5.0 Storage Features Part 1 – VMFS-5 you might be pressured to believe that it’s all good, caveats aside that your VMFS5 upgraded datastores will rock out just like VMFS3, but take a gander at Jason Boche @JasonBoche VMFS-5 VMFS-3, What’s the Deal?

Differences between upgraded and newly created VMFS-5 datastores:

VMFS-5 upgraded from VMFS-3 continues to use the previous file block size which may be larger than the unified 1MB file block size. Copy operations between datastores with different block sizes won’t be able to leverage VAAI.  This is the primary reason I would recommend the creation of new VMFS-5 datastores and migrating virtual machines to new VMFS-5 datastores rather than performing in place upgrades of VMFS-3 datastores.
VMFS-5 upgraded from VMFS-3 continues to use 64KB sub-blocks and not new 8K sub-blocks.
VMFS-5 upgraded from VMFS-3 continues to have a file limit of 30,720 rather than the new file limit of > 100,000 for newly created VMFS-5.
VMFS-5 upgraded from VMFS-3 continues to use MBR (Master Boot Record) partition type; when the VMFS-5 volume is grown above 2TB, it automatically switches from MBR to GPT (GUID Partition Table) without impact to the running VMs.
VMFS-5 upgraded from VMFS-3 will continue to have a partition starting on sector 128; newly created VMFS-5 partitions start at sector 2,048.


Hey, calm down, calm down… I think we can solve this problem pretty easily! And if the burning sensation continues, get that checked out!

I don’t know about you, you might have 1 vCenter, 3 Servers and a handful of Datastores. I have thousands… of EACH, so I needed something to do my scans and checks at scale with PowerCLI because I’m a baller, and apparently I pull that off rather well. :)

Methods of detecting whether you have VMFS3, VMFS3 upgraded to VMFS5 or otherwise mismatched sets..

  • Block Size is greater than 1MB
  • Partition type is msdos instead of gpt (*Detected by checking whether the StartSector is 128 instead of 2048)
  • And of course, your VMFS version is VMFS3 or 3.46

But wait, didn’t you say above that partition types will change from MSDOS to GPT if they’re expanded? Doesn’t that make tracking harder? Yes.

Let’s get our PowerCLI on so we can crack this nut!

Get-Datastore | Get-View | Select-Object Name,@{N="VMFS version";E={$_.Info.Vmfs.Version}},@{N="BlocksizeMB";E={$_.Info.Vmfs.BlockSizeMB}}
// To check VMFS version & block sizes – Really useful to just check in general that your version is 5.54 and your blocksize is 1MB though if you VMFS3->5 In-place upgrade when VMFS3 was 1MB, this won’t reveal itself to you

Get-Datastore | Get-View | Where {$_.Info.Vmfs.Version –eq “3.46”} | Where {$_.Info.Vmfs.BlockSizeMB -eq "1"} | Select-Object Name,@{N="VMFS version";E={$_.Info.Vmfs.Version}},@{N="BlocksizeMB";E={$_.Info.Vmfs.BlockSizeMB}}
// To Check VMFS Version and Block Sizes but only listing mismatches you specify  – so if you want to see if you specifically have any 3.46 VMFS and various block sizes

Busting out the mad $esxcli syntax!

This is where we start to get real. The following examples are simple ‘one-liners’ which are cute and all if you’re checking one host, but I also give you one which will scan EVERYTHING so you can just sit back and bask in the glow of figuring out WTF IS GOING ON WITH THIS BURNING, HELP HELP HELP!

$ | Select Device, StartSector
// Dumps all Offsets – This can be useful if you want to see a lot of data… but less so if you’re looking for something specific…

$ | group-Object -Property Device | Where {$_.StartSector –eq “128”} | Select Device, StartSector
// Dumps only offsets which "equal" a startsector, in this case 128 – Now we’re cooking with oil, a StartSector of 128 leans on a datastore being VMFS3 or VMFS5 which had been upgraded from VMFS3

$ | Where {$_.StartSector -eq "128"} | Select Device, StartSector
// This will dump all of your partitions which have a starting offset of 128, same as above but shorter

Script me baby one more time!

OMG YOU JUST MADE A HORRIBLY DATED REFERENCE TO BRITNEY SPEARS. For what its worth, I believe she did some scripting in her days…   The scenarios below will just ‘do it’ based upon whatever hosts you connected to with Connect-VIServer, obviously the difference being ‘comments’ or not.

foreach ($myHost in get-VMHost)
#This tells the system to do a run the command against all "VMHosts" that you have defined as part of your Connect-VIServer
    Write-Host ‘$myHost = ‘ $myHost
    #Display the ESXi Host that it is operating against, helps if you’re scanning multiple vCenters
    $esxcli = Get-EsxCli -VMHost $myHost
    #This sets the syntax and the context for the Get-EsxCli command to operate, a requirement for running $esxcli.Commands
    $ |
    #Use Get-EsxCli to list the core storage devices
    Where {$_.StartSector -eq "128"} |
    #This specifies we’re only looking for partitions which have a StartSector of 128, which could mean either VMFS3 or VMFS3 upgraded to VMFS5 Datastores
    Select Device, StartSector
    #When all is said and done, it’s nice to see it in a ‘pretty’ format to see what work you need to do!

Without Comments

foreach ($myHost in get-VMHost)
    Write-Host ‘$myHost = ‘ $myHost
    $esxcli = Get-EsxCli -VMHost $myHost
    $ |
    Where {$_.StartSector -eq "128"} |
    Select Device, StartSector

Now technically you could use partedUtil but that’s a pain in the ass.  – But for the sake of continuity here is the syntax/results!

~ # partedUtil getptbl "/vmfs/devices/disks/naa.60a98000646e4f4b475a6a4975422d66"
261083 255 63 4194304000
1 128 4194298394 251 0
~ # partedUtil getptbl "/vmfs/devices/disks/naa.60a98000646e4f4b475a70516f34416f"
534698 255 63 8589934592
1 2048 8589934558 AA31E02A400F11DB9590000C2911D1B8 vmfs 0
~ #

So in case you’re wondering if the script is working properly you should end up with results similar to this below;

$myHost =  103.domain.local

Device                                                      StartSector
——                                                      ———–
naa.60a98000323764703424434e6246775a                        128
$myHost =  102.domain.local
naa.60a98000323764703424434e6246775a                        128
$myHost =  101.domain.local
naa.60a98000323764703424434e6246775a                        128
$myHost =  037.domain.local

And that is basically all it takes! This hopefully should give you the fuel you need to scan your environment with minimal effort and identify any VMFS3 datastores so you can clean that stuff up and MIGRATE!  I discovered a bunch of them which aren’t so kind, and what ensues is massive migrations!

Good luck out there! and if you can find some good way to hack esxcli to ALSO have it then correlate that data to what the datastore name is… I FAILED :)

Tags: , , , , , , , , ,
Posted in PowerCLI, Storage, Virtualization, vmware, vSphere | Comments (3)

VMware Log Insight EXPOSED! Splitting your Syslog with an axe!

June 17th, 2013

Well, for those of you who have read my “Exposed” expose’ as in the past… I’ll do my best to provide an in-depth coverage of this tool, lessons learned and so much more!  Allow me to disclaimer for a moment, this IS a beta, so your mileage may vary and your feedback has a chance to shape the product.

You can read the infamous Jon Herlocker’s breakdown of the tool at the Office of the CTO Blog; Introducing VMware vCenter Log Insight

Jon provides some great stock photos, descriptions, images, use-cases and all that jazz… What I’ll show you, is Production Use, and I won’t be using any screenshots I didn’t take myself! :)

Getting Started with #LogInsight

WTF YOU’RE ALREADY HASHTAGGING IT! Yea I am, but I digress. :)  Alright! Let’s focus on getting started!   First things first, you should visit the VMware Log Insight Beta Community – There you can join the ‘discussion forums’ okay, I know you won’t seriously do that, but you can download the product!

And once you get it downloaded and you deploy the OVA/OVF you’re pretty much set! You may experience ‘errors’ when going through the configuration process, I personally re-deployed my OVF 3 times (remember, it’s a beta) but once I got past that and little browser mess-ups, it’s been SOLD since!

Login Page

Look at that, nice clean login… seems pretty straight-forward (hint… it is :))

Cracking Open the Log (insight…)

I know what you’re saying DAMNIT MORE BAD LOG PUNS. Yea, that’s right! Alright, so you pop it open and here’s your dashboard!   You’ll notice events coming in, very simple interface, perhaps too simple but simple nonetheless.   The real keys will come into the next few sections.




Once you start diving into the details you’ll start to see more and more events coming in, and in their relevant and relative categories.


I want to share with you this little experience… Sometimes you may click on a tab and be all like WTF HAPPENED THERE WERE EVENTS HERE 5 MINUTES AGO. And that’s exactly it.  If you’re on the “Last 5 minutes of data” section, you’re literally only going to get the last 5 minutes of data.  Expand it out to an hour or so and you’ll start to see those messages you had seen just minutes before! 

vCenter_Servers Events_Tasks_Alarms

And lastly your main page happens to list again further various event types of screens… And I get it, this is all nice and interesting, but what does it mean?!

Diving into the weeds


Once you start to get into the “Interactive Analysis” you start to get into the details, or quite frankly into the damn SYSLOGs!


One particularly awesome piece of this is the ability to ‘type’ something into the Search bar.  What this does is, it indexes all types of requests in the background and gives you an idea of how many of certain types of events or names shows up.  For example, if you specify a Hostname you’ll see how many syslog messages had that hostname, or VM Guest, or you name it.  Just type something in, and you’ll start to get some details and insight! (For security reasons.. I chose details which had no particular relevance but still provided you some ‘search’ context!)

Configuring your ESX environment!

You may notice upon reading the manuals which come with the software (hah, you’re never going to read those! ;)) but it comes with a tool called ‘Configure ESXi’ which will configure your environment.  Let’s say you’re like me and cannot run that tool, or just outright choose not to… Well, here are some alternatives to get your ESX hosts configured so they can start reporting back to your newly created SYSLOG Server!


That’s right.  You find that your local traffic is okay, but you have a remote site which has a slower link, could be in a different country, or just over a Satellite like or something similarly ridiculous… Well, look no further!

When using VMware Log Insight you may want to change the amount of SysLog data you’re receiving

You can check your current logging levels with this PowerCLI command(s)

  • Get-VMHost | Get-VMHostAdvancedConfiguration -Name "Config.HostAgent.log.level"
  • Get-VMHost | Get-VMHostAdvancedConfiguration -Name "Vpx.Vpxa.config.log.level"

Chances are you’ll be getting a load of data coming in at Verbose ~1000s, even as high as 5000-10,000 logs in a 5 minute period.

I switched hostagent and vpx levels from "Verbose" to "Warning" and went down to ~10-15 logs for a 5 minute span!    If you have low bandwidth links this could mean significantly less impact.

And if you want to outright change those down to Warning as I did, or to any other value (say, Info) you can do it with these handy one-liners!

  • Get-VMHost | Set-VMHostAdvancedConfiguration -Name "Config.HostAgent.log.level" -Value "warning"
  • Get-VMHost | Set-VMHostAdvancedConfiguration -Name "Vpx.Vpxa.config.log.level" -Value "warning"


I’m glad you asked that, I mean metaphorically of course, because I’m writing this, not you! NOT YOU!   I went through various iterations to make this possible, and I found setting the Syslog server easy, Configuring the Firewalls equally easy whether via vSphere Client or PowerCLI, but I found the reloading the syslogd to be a pain in the ass.  That is until I came across this little gem!

I’d like to note I am stealing/borrowing this from Caleb in his post; Changing VMware ESXi 5.1 Syslog settings via PowerCLI – It worked like a charm and you shouldn’t feel shamed to use it!   Be sure you thank Caleb for this code of course!

    • get-vmhost | Get-VMHostAdvancedConfiguration -Name
    • #Get Each Host Connected to the vC
    • foreach ($myHost in get-VMHost)
    • {
    •     #Display the ESXi Host that you are applying the changes to
    •     Write-Host ‘$myHost = ‘ $myHost
    •     #Set the Syslog LogHost
    •     Set-VMHostAdvancedConfiguration -Name -Value ‘,’ -VMHost $myHost
    •     #Use Get-EsxCli to restart the syslog service
    •     $esxcli = Get-EsxCli -VMHost $myHost
    •     $esxcli.system.syslog.reload()
    •     #Open the firewall on the ESX Host to allow syslog traffic
    •     Get-VMHostFirewallException -Name "syslog" -VMHost $myHost | set-VMHostFirewallException -Enabled:$true
    • }

And honestly, that is about it! Once you’re set with the right level of verbosity of information, and syslogs pointing to your newly built VMware Log Insight server… then it’s just a matter of collecting, and reviewing with the occasional troubleshooting as needed.  

I did come across this little bug which I’m sure they’ll fix eventually..


If you’re not seeing the bug, it simply is, if you have the Log Insight log viewer NOT in full-screen mode (like you have half the screen showing log insight, and the other half, oh I don’t know… with VLC finishing off the 4th season of Battlestar Galactica…) it’ll seemingly ‘truncate’ the text on the screen, instead of simply moving to the next line.   I’m sure it’d be pretty easy to fix, so don’t get too annoyed by it! :)

In Summation or Building a Log Cabin for your troubleshooting…

Wow, you couldn’t let it go without another bad pun? Yea, probably not… :)   There is a lot more to this tool than I could show you, unfortunately there are screens… which I could not edit down enough without destroying the value of what you’d be seeing.   This tool has vCenter Operations integration, the ability to pull and index all of your data points! I can see at a glance errors which are showing up, and then drill-down to find similarly correlated errors.   I mean, the tool isn’t overly too intelligent yet, but that is bound to come in time, and through our suggestions I hope!

I encourage you to check it out, especially if you don’t have something in place pulling your syslogs today, like Kiwi or Splunk.   This gives you a ‘single family’ set of solutions which in the end will have your virtual best interests at heart.   So check out the beta and let me know what you think.   I’ll keep rocking this tool out and continuing to pull in and index my extremely enormous virtual environment!   Enjoy!

Tags: , , , , , , , , ,
Posted in vCenter Operations Manager, vCOPS, virt, Virtualization, vmware | Comments (7)

  • Archives