Jekyll2022-12-13T23:06:56+00:00https://devmeetsops.com//feed.xmlDev Meets OpsBrian MarshCross-Platform Certificate Encrypted Strings2018-04-16T00:00:00+00:002018-04-16T00:00:00+00:00https://devmeetsops.com//encryption/cross-platform-cms-messages<p>Password vaulting solves a very real problem: how do we leverage automation and avoid hardcoding passwords in source? We can use products like Hashicorp’s Vault to provide RBAC, password expirations, and other advanced functionality that can plug into automation but this introduces a new problem - how do we protect that “first secret” that allows the newly built server to access and authenticate with the vault? We could set this password in plaintext/source - but that’s the problem that vaulting attempts to solve.</p>
<p>One potential solution is to leverage CMS messages. The <a href="https://en.wikipedia.org/wiki/Cryptographic_Message_Syntax">Cryptographic Message Syntax</a> is a standardized method for signing, encrypting, decrypting, and validating any form of digital data using certificate-based key pairs.</p>
<p>A certificate must have extensions and usage that specifies document encryption in order decrypt/encrypt CMS messages. On Windows, this certificate resides in a certificate store on the local system - this certificate store is protected with a combination of ACL and the Windows Data Protection API. On Linux, the private key can be protected via folder and file permissions.</p>
<p>If the existing VM automation relies upon vRA, vRO, binary artifact repository and source repository - one potential usage for CMS messages to solve the “First Secret” problem is as follows:</p>
<ul>
<li>A new VM either has the certificate burned in to the template, or the certificate can be stored in vRO and injected into VMs at deployment time.</li>
<li>The CMS message containing the “first secret” can be made available in source or binary repositories.</li>
<li>At the opportune time, the VM will retrieve the CMS message, decrypt with the certificate and use this credential to auth with the desired password vaulting software.</li>
<li>After the VM successfully authenticates and before the VM is handed off, vRO can forcibly remove the certificate from the VM to reduce exposure.</li>
</ul>
<h2 id="create-the-certificate-windows--powershell">Create the Certificate (Windows / PowerShell)</h2>
<ul>
<li>Manually create certificate with an inf file and certreq:
<ul>
<li>Create certreq.inf file for cert request (example below). Most things can be changed, but <strong><code class="language-plaintext highlighter-rouge">szOID_ENHANCED_KEY_USAGE</code></strong>, <strong><code class="language-plaintext highlighter-rouge">szOID_DOCUMENT_ENCRYPTION</code></strong>, <strong><code class="language-plaintext highlighter-rouge">KeyUsage</code></strong>, and the <strong><code class="language-plaintext highlighter-rouge">[Extensions]</code></strong> fields should be left as is.
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>[Version]
Signature = "$Windows NT$"
[Strings]
szOID_ENHANCED_KEY_USAGE = "2.5.29.37"
szOID_DOCUMENT_ENCRYPTION = "1.3.6.1.4.1.311.80.1"
[NewRequest]
Subject = "cn=certenc@mydomain.com"
MachineKeySet = false
KeyLength = 8192
KeySpec = AT_KEYEXCHANGE
HashAlgorithm = Sha1
Exportable = true
RequestType = Cert
KeyUsage = "CERT_KEY_ENCIPHERMENT_KEY_USAGE | CERT_DATA_ENCIPHERMENT_KEY_USAGE"
ValidityPeriod = "Years"
ValidityPeriodUnits = "1"
[Extensions]
%szOID_ENHANCED_KEY_USAGE% = "{text}%szOID_DOCUMENT_ENCRYPTION%"
</code></pre></div> </div>
</li>
<li>
<p>Create a new private/public key pair:</p>
<p><code class="language-plaintext highlighter-rouge">C:\> Certreq -new c:\path\to\certreq.inf output.cert</code></p>
</li>
</ul>
</li>
<li>Alternatively, create it with powershell:
<pre><code class="language-language-powershell"> C:\> New-SelfSignedCertificate -Subject "CN=certenc@mydomain.com" -CertStoreLocation "Cert:\CurrentUser\My" -KeyUsage KeyEncipherment,DataEncipherment, KeyAgreement -Type DocumentEncryptionCert
</code></pre>
</li>
</ul>
<h2 id="export-certificate-to-linux">Export Certificate to Linux</h2>
<ul>
<li>Export the certificate and private key pair with PowerShell
<pre><code class="language-language-powershell">C:\> $MyPassword = ConvertTo-SecureString -String "exportpassword" -Force -AsPlainText
C:\> (Get-ChildItem -path Cert:\CurrentUser\My\).Where({$_.Subject -eq "CN=certenc@mydomain.com"}) | Export-PfxCertificate -FilePath c:\temp\mypfx.pfx -Password $MyPassword
</code></pre>
</li>
<li>On the Linux server, convert the pfx and remove the password
<ul>
<li>Convert the entire pfx to a pem file
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>~/> openssl pkcs12 -in mypfx.pfx -out all.pem
Enter Import Password: [exportpassword]
MAC verified OK
Enter PEM pass phrase: [exportpassword]
Verifying - Enter PEM pass phrase: [exportpassword]
</code></pre></div> </div>
</li>
<li>Export key into rsa format
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>~/> openssl rsa -in all.pem -out rsa.key
Enter pass phrase for all.pem: [exportpassword]
writing RSA key
</code></pre></div> </div>
</li>
<li>This will result in two files: <code class="language-plaintext highlighter-rouge">all.pem</code> and <code class="language-plaintext highlighter-rouge">rsa.key</code>.</li>
</ul>
</li>
</ul>
<h2 id="encryption">Encryption!</h2>
<ul>
<li>Encrypt your secret:
<ul>
<li>Windows
<ul>
<li>Command:
<pre><code class="language-language-powershell">C:\> Protect-CmsMessage -Content "SecretPassword1!" -To *certenc@mydomain.com
-----BEGIN CMS-----
MIIBuQYJKoZIhvcNAQcDoIIBqjCCAaYCAQAxggFRMIIBTQIBADA1MCExHzAdBgNVBAMMFmNlcnRl
bmNAbWFzdGVyY2FyZC5jb20CEDtwj/MAVhipS6fXV2mKPRgwDQYJKoZIhvcNAQEHMAAEggEAqdy6
BJ3F2zkCw0s1wwRXyfZ6LO5C/DldszJi9o5jodwh75pmbCkd+sOfU2XwIVSfbzYdgEuwpKqpoDFJ
ykbuskBI9zqCXEDrWLSksRr9UfBdM4lToZR58IIhKhe7d6PU1YCvXcGM2zF2MzE1lPJHl8q9O24c
EHxvz1OHaVcLnhZ+bxOfOhTbidiEr5DwTrL69gChWCAf4Yv6EUzacEsO08+A0kvWRBiwmtBSnprl
6mJW4pgPZN6qgxSBJafswv5AS3X+9/bIyNoyNIwD3o5ZcEJ5CtZa2YGW6P1n1EMyOSeDndC8sxj1
VWydE3aJ6aZwcI0P6Ifv38wjUNV/Wbs/HzBMBgkqhkiG9w0BBwEwHQYJYIZIAWUDBAEqBBAPgGSD
tSj98L7jJLdlRpTxgCB6BfCAEwR8hQx9CyfbpfRTqgEFLsxHn76IgRWUvA5e8A==
-----END CMS-----
</code></pre>
</li>
<li>The output can be captured to a plaintext file by adding the Out-File cmdlet:
<pre><code class="language-language-powershell">C:\> Protect-CmsMessage -Content "SecretPassword1!" -To *certenc@mydomain.com | Out-File -Path C:\Temp\encrypted.enc
</code></pre>
</li>
</ul>
</li>
<li>Linux
<ul>
<li>Command:
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>~/> openssl cms -encrypt -from demo@localhost -to certenc@mydomain.com -subject "encrypted" -out encrypted.enc -outform PEM all.pem
SecretPassword1!
[ctrl]-[d]
</code></pre></div> </div>
</li>
<li>This results in a mail.msg file like below
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>~/> cat encrypted.enc
-----BEGIN CMS-----
MIIBqAYJKoZIhvcNAQcDoIIBmTCCAZUCAQAxggFRMIIBTQIBADA1MCExHzAdBgNV
BAMMFmNlcnRlbmNAbWFzdGVyY2FyZC5jb20CEDtwj/MAVhipS6fXV2mKPRgwDQYJ
KoZIhvcNAQEBBQAEggEABPSASeNou8YBTjaEe3s+buvZwJuw7pGpLh3Hu2bsve+b
+gTzoZuPAeLPR1LhAQVJiBXxSJUyGNemviehSgndOKYFWhojQnnncB/i/nin5ijk
uBzuXgcFBEJDvFqtD8IsXGkLdwa2D7nuELpqd5rZXGE9GjMSFVtkK2gsczKE1Rza
URVNGcWt94QgPQvx4xk1svp7wRgkBIOi+1nmXQ5r1kdS70gz6Z1pV9xanu5KhPyb
fn8EwyzYoNiEsheFaSzstJCUOYiQckXjY/i/aSezrYM9PDPefCZqIIsTLQlAbi+c
qhN5jPs4utMVpFDnT5oHG8ObsJQjGFKmpjbKRiWv8DA7BgkqhkiG9w0BBwEwFAYI
KoZIhvcNAwcECEjEr1W2znlsgBgwF9+IDrrHEqStVkMkCMrO/PnpcM5dLIY=
-----END CMS-----
</code></pre></div> </div>
</li>
</ul>
</li>
</ul>
</li>
</ul>
<h2 id="decryption">Decryption!</h2>
<ul>
<li>Decrypt your secret
<ul>
<li>Windows
<ul>
<li>Obtain or create a file with the CMS message content
<pre><code class="language-language-powershell">C:\> $Content = @"
>> -----BEGIN CMS-----
>> MIIBqAYJKoZIhvcNAQcDoIIBmTCCAZUCAQAxggFRMIIBTQIBADA1MCExHzAdBgNV
>> BAMMFmNlcnRlbmNAbWFzdGVyY2FyZC5jb20CEDtwj/MAVhipS6fXV2mKPRgwDQYJ
>> KoZIhvcNAQEBBQAEggEABPSASeNou8YBTjaEe3s+buvZwJuw7pGpLh3Hu2bsve+b
>> +gTzoZuPAeLPR1LhAQVJiBXxSJUyGNemviehSgndOKYFWhojQnnncB/i/nin5ijk
>> uBzuXgcFBEJDvFqtD8IsXGkLdwa2D7nuELpqd5rZXGE9GjMSFVtkK2gsczKE1Rza
>> URVNGcWt94QgPQvx4xk1svp7wRgkBIOi+1nmXQ5r1kdS70gz6Z1pV9xanu5KhPyb
>> fn8EwyzYoNiEsheFaSzstJCUOYiQckXjY/i/aSezrYM9PDPefCZqIIsTLQlAbi+c
>> qhN5jPs4utMVpFDnT5oHG8ObsJQjGFKmpjbKRiWv8DA7BgkqhkiG9w0BBwEwFAYI
>> KoZIhvcNAwcECEjEr1W2znlsgBgwF9+IDrrHEqStVkMkCMrO/PnpcM5dLIY=
>> -----END CMS-----
>> "@
</code></pre>
</li>
<li>Decrypt with the Unprotect-CmsMessage cmdlet
<code class="language-plaintext highlighter-rouge">language-powershell
C:\> Unprotect-CmsMessage -Content $content
SecretPassword1!
</code>`</li>
</ul>
</li>
<li>Linux
<ul>
<li>Obtain or create a file with the CMS message content
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>~/> echo "
-----BEGIN CMS-----
MIIBuQYJKoZIhvcNAQcDoIIBqjCCAaYCAQAxggFRMIIBTQIBADA1MCExHzAdBgNVBAMMFmNlcnRl
bmNAbWFzdGVyY2FyZC5jb20CEDtwj/MAVhipS6fXV2mKPRgwDQYJKoZIhvcNAQEHMAAEggEAqdy6
BJ3F2zkCw0s1wwRXyfZ6LO5C/DldszJi9o5jodwh75pmbCkd+sOfU2XwIVSfbzYdgEuwpKqpoDFJ
ykbuskBI9zqCXEDrWLSksRr9UfBdM4lToZR58IIhKhe7d6PU1YCvXcGM2zF2MzE1lPJHl8q9O24c
EHxvz1OHaVcLnhZ+bxOfOhTbidiEr5DwTrL69gChWCAf4Yv6EUzacEsO08+A0kvWRBiwmtBSnprl
6mJW4pgPZN6qgxSBJafswv5AS3X+9/bIyNoyNIwD3o5ZcEJ5CtZa2YGW6P1n1EMyOSeDndC8sxj1
VWydE3aJ6aZwcI0P6Ifv38wjUNV/Wbs/HzBMBgkqhkiG9w0BBwEwHQYJYIZIAWUDBAEqBBAPgGSD
tSj98L7jJLdlRpTxgCB6BfCAEwR8hQx9CyfbpfRTqgEFLsxHn76IgRWUvA5e8A==
-----END CMS-----
" > encrypted.enc
</code></pre></div> </div>
</li>
<li>Decrypt with the openssl cms executable
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>~/> openssl cms -decrypt -in encrypted.enc -recip all.pem -inkey rsa.key -inform PEM
SecretPassword1!%
</code></pre></div> </div>
</li>
<li>Note that openssl seems to add a ‘<code class="language-plaintext highlighter-rouge">%</code>’ to the end of the string.</li>
</ul>
</li>
</ul>
</li>
<li>This string can now be used in scripts/credentials/other operations</li>
</ul>Brian MarshPassword vaulting solves a very real problem: how do we leverage automation and avoid hardcoding passwords in source? We can use products like Hashicorp’s Vault to provide RBAC, password expirations, and other advanced functionality that can plug into automation but this introduces a new problem - how do we protect that “first secret” that allows the newly built server to access and authenticate with the vault? We could set this password in plaintext/source - but that’s the problem that vaulting attempts to solve.Pipeline Mock-up2018-03-30T00:00:00+00:002018-03-30T00:00:00+00:00https://devmeetsops.com//source%20control/pipeline<p>I have been toying with setting up a CI/CD Pipeline for PowerShell module development. Basic components and steps are as follows:</p>
<h2 id="components">Components:</h2>
<ul>
<li>Git Source Control</li>
<li>Orchestration (Jenkins, TeamCity, etc)</li>
<li>Binary repository (Artifactory, PowerShell Gallery, etc)</li>
</ul>
<h2 id="steps">Steps:</h2>
<ol>
<li>Code checked into a git-based source control. <code class="language-plaintext highlighter-rouge">master</code> branch is “Production Ready”, <code class="language-plaintext highlighter-rouge">release/stage</code> is “Ready for QA”, other branches are considered in development.</li>
<li>Orchestration monitors the repo for pull requests on <code class="language-plaintext highlighter-rouge">release/stage</code>. PRs trigger automatic tests bundled with the code in a predetermined folder path.</li>
<li>Successful tests are reported … Somehow.</li>
<li>When a PR is approved/merged, Orchestration re-runs tests (for validation) and merges the code into the <code class="language-plaintext highlighter-rouge">master</code> branch.</li>
<li>When the <code class="language-plaintext highlighter-rouge">master</code> branch is updated, Orchestration builds the module (process TBD) and pushes to binary repository.</li>
<li>(Optional) When code is run, check for updates from the binary repository before first run, updating the local code to the latest version as necessary.</li>
</ol>
<p>For git, I’m using <a href="https://gogs.io">Gogs</a> - it’s simple to set up and is the platform for my personal/private repo. I do not have a running Jenkins instance that is internet accessible, I’m debating splitting the DigitalOcean droplet - letting it do double duty: Gogs and Jenkins. Alternatively, I can spin up a vagrant box with Jenkins installed/pre-configured to monitor Gogs. To accomplish steps 2-5, I’ll need three Jenkins jobs:</p>
<table>
<thead>
<tr>
<th>Stage</th>
<th>Description</th>
<th>Trigger</th>
<th>Action</th>
</tr>
</thead>
<tbody>
<tr>
<td>1</td>
<td>Monitor for PRs and test</td>
<td>PR Request</td>
<td>Run tests and report</td>
</tr>
<tr>
<td>2</td>
<td>Monitor <code class="language-plaintext highlighter-rouge">release/stage</code> for updates and merge with prod</td>
<td>Branch update</td>
<td>Run tests and merge</td>
</tr>
<tr>
<td>3</td>
<td>Monitor <code class="language-plaintext highlighter-rouge">master</code> for updates and publish</td>
<td>Branch update</td>
<td>Run tests, package, and push to binary repo</td>
</tr>
</tbody>
</table>
<p>To handle the optional 6th step, one option would be something like <a href="https://github.com/RamblingCookieMonster/PSDepend">PSDepend</a> (something I’ve been meaning to take a closer look at in the future). I am lucky enough to have distinct Dev, QA, and Prod systems from which my PowerShell code can be run. Moreover, each of the higher-level orchestration engines that calls PowerShell is somewhat self aware - it knows what environment in which the orchestration workflow is running. If I own the binary repositories, I can configure multiple repositories for the configured environment:</p>
<ul>
<li>Dev Orchestration == Dev Binary Repository</li>
<li>QA Orchestration == QA Binary Repository</li>
<li>Prod Orchestration == Prod Binary Repository</li>
</ul>
<p>Using this information, I can query the corresponding binary repo for a set of required modules and compare that to my local set/versions. If the local copy is outdated, I can pull down the latest version. Once the local version has been updated, I can load that specific version of the module and proceed with my normal PowerShell code execution.</p>
<p>The code necessary for checking local versions against a named remote repo can be found in the gist below. A future post will continue this process and describe how I can automatically register specific repositories based on environment if they do not already exist.</p>
<script src="https://gist.github.com/e255ee928278b5346e5a91c84d7d11b4.js"> </script>Brian MarshI have been toying with setting up a CI/CD Pipeline for PowerShell module development. Basic components and steps are as follows:Using CloudClient to bulk change vRA Reservations2017-12-12T00:00:00+00:002017-12-12T00:00:00+00:00https://devmeetsops.com//automation/cloudclient-vra-reservations<p>What happens when you have ~500 VMs in vRealize Automation that need to have their reservations changed? You figure out a way to script that tedious process. (This will provide some background and more details on a recent <a href="https://code.vmware.com/forums/5098/vrealize-automation#577825">VMware Communities</a> post.)</p>
<p>I first started looking at what I was most familiar with - <a href="https://github.com/jakkulabs/PowervRA">PowervRA</a> - a great community supported, PowerShell wrapper for the vRA API stack. Unfortunately, PowervRA does not offer reservation manipulation - only reporting.</p>
<p><a href="https://code.vmware.com/web/dp/tool/cloudclient/4.5.0">CloudClient</a> is a Java application wrapper for the vRA API. Bulk changing reservations is one use case, but several other uses exist in the documentation from that download site.</p>
<p>To use CloudClient in a wrapper script, we must first save our password by interactively launching CloudClient and executing the <code class="language-plaintext highlighter-rouge">login keyfile</code> command. This command specifies an output location for the encrypted file and prompts for a password. Most CloudClient activities require authentication with both vRA and vRA’s IaaS component.
<img src="/images/posts/CloudClient_Login.png" alt="CloudClient" /></p>
<p>With the files saved, I can then create a CloudConfig.properties file
<script src="https://gist.github.com/7852c15bf46f9deb3c9f6c681740420a.js"> </script></p>
<p>With the CloudConfig.properties file in place at the same level as the <code class="language-plaintext highlighter-rouge">./bin</code> folder, I can execute my commands directly from PowerShell</p>
<figure class="highlight"><pre><code class="language-powershell" data-lang="powershell"><span class="o">.</span><span class="n">\bin\cloudclient.sh</span><span class="w"> </span><span class="nx">vra</span><span class="w"> </span><span class="nx">machines</span><span class="w"> </span><span class="nx">change</span><span class="w"> </span><span class="nx">reservation</span><span class="w"> </span><span class="nt">--ids</span><span class="w"> </span><span class="s2">"restest01"</span><span class="w"> </span><span class="nt">--reservationName</span><span class="w"> </span><span class="s2">"Windows\ -\ 02"</span></code></pre></figure>
<p>This works great for single VMs, but for migrating several hundred - and ensuring the VMs you’re requesting are indeed on the wrong reservation takes a bit more effort. If I connect to vRA with PowervRA, I can gather a list of VMs on the “old” reservation and pass that list of applicable servers to the CloudClient command line. Alternatively, if I provide a list of VMs to PowerShell, I can validate which are on the old reservation and update only those.</p>
<p>The final source can be found below:
<script src="https://gist.github.com/479cdcd4609b03b58c3fbfb2df3f30f5.js"> </script></p>Brian MarshWhat happens when you have ~500 VMs in vRealize Automation that need to have their reservations changed? You figure out a way to script that tedious process. (This will provide some background and more details on a recent VMware Communities post.)Finalizing the lab infrastructure2016-10-12T00:00:00+00:002016-10-12T00:00:00+00:00https://devmeetsops.com//home%20lab/finalizing-lab-infrastructure<p>In order for my home lab setup to be finalized, I needed to do two things:</p>
<ul>
<li>I need to provide “jumpbox” capabilities - with the self-imposed restriction that no ssh/rdp ports will be opened to the outside world.</li>
<li>CoreOS IP Schema (Public/Private)</li>
<li>Setup port forwarding for ports 80 & 443 to the CoreOS cluster</li>
</ul>
<h2 id="jumpbox">Jumpbox</h2>
<p>I’m using <a href="https://guacamole.incubator.apache.org/">guacamole</a> to provide this access. As previously stated, I’m isolating my homelab networks from the everything else with rules in my Edgerouter Lite, so the guacamole VM resides inside of the home lab network segment. I’m forwarding to port 8443 to free up the standard 443/80 for the CoreOS cluster.</p>
<h2 id="coreos-ip-schema">CoreOS IP Schema</h2>
<p>To better mirror how CoreOS works in the public cloud, I need to set a public and private IP for each of my CoreOS instances. It took some tweaking, but now my PowerShell Script will deploy CoreOS with “public” IP addresses on VLAN 30, and “private” IP addresses on VLAN 40. The etcd service and fleet runs on VLAN 40 while the docker services are running on VLAN 30.</p>
<h2 id="coreos-port-forwarding">CoreOS Port Forwarding</h2>
<p>I’m forwarding to an HAProxy instance in the home lab segment - I’ve tested a bit with SSL pass through in the past (I’m sure that’ll come up again in the future). HAProxy’s check options will cover the cases when services are running on different nodes.</p>Brian MarshIn order for my home lab setup to be finalized, I needed to do two things:Adventures in CoreOS: The Saga Continues2016-10-05T00:00:00+00:002016-10-05T00:00:00+00:00https://devmeetsops.com//automation/adventures-in-coreos-the-saga-continues<p>I’ve worked with <a href="http://blog.briankmarsh.com/adventures-with-coreos/">CoresOs</a> in the <a href="http://blog.briankmarsh.com/adventures-with-coreos-part-2/">past</a>, but thought that two years was enough of a gap before I attempted to deal with it again.</p>
<p>In my original blog post, I had to handcraft individual ISOs for each CoreOS instance - this was painful and slow. Luckily, things have changed (for the better). CoreOS can now take advantage of VMware Tools and leverage properties present in the <a href="https://coreos.com/os/docs/latest/vmware-guestinfo.html">VM config file</a>. With that knowledge, (and <a href="https://robertlabrie.wordpress.com/2015/09/27/coreos-on-vmware-using-vmware-guestinfo-api/">this blog post</a> in hand), I opted to spin up a new three node cluster in my home lab.</p>
<h1 id="the-script">The Script</h1>
<p>I’ve built upon the original script from Robert Labrie, doing a bit of hard coding (my vCenter instance is temporary and still uses the default password for the administrator@vsphere.local account). The final result can be found on <a href="https://github.com/pezhore/vmware_coreos">GitHub</a>.</p>
<p>Given a specified quantity of nodes, and a source template, several CoreOS VMs are deployed in vCenter. After the VMs are deployed, a cloud-config file must be provided to provide configuration for the new VMs. I’m leveraging a source yaml file for the bulk of the CoreOS configuration with specific information (networking, discovery URL, etc) generated by PowerShell script at run time. This configuration data is encoded with base64 and added to the VM’s vmx config file.</p>
<h1 id="helpful-tidbits">Helpful Tidbits</h1>
<p>This bit requests a new discovery url with the given node count and updates the cloud-config.yml.</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># Get a etcd cluster discovery url with the given size</span><span class="w">
</span><span class="nv">$req</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Invoke-WebRequest</span><span class="w"> </span><span class="nt">-Uri</span><span class="w"> </span><span class="s2">"https://discovery.etcd.io/new?size=</span><span class="nv">$NodeCount</span><span class="s2">"</span><span class="w">
</span><span class="c"># Get the current cloud-config content, then replace the existing discovery line with the new discovery url</span><span class="w">
</span><span class="c"># This is kludgey and should probably be replaced by yaml manipulation</span><span class="w">
</span><span class="n">get-content</span><span class="w"> </span><span class="o">.</span><span class="nx">\cloud-config.yml</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">ForEach-Object</span><span class="w"> </span><span class="p">{</span><span class="bp">$_</span><span class="w"> </span><span class="o">-replace</span><span class="w"> </span><span class="s2">"discovery: .*"</span><span class="p">,</span><span class="w"> </span><span class="s2">"discovery: </span><span class="si">$(</span><span class="nv">$req</span><span class="o">.</span><span class="nf">Content</span><span class="si">)</span><span class="s2">"</span><span class="p">}</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">Set-Content</span><span class="w"> </span><span class="o">.</span><span class="nx">\cloud-config.yml</span><span class="w">
</span></code></pre></div></div>
<p>In order to allow variable size clusters, I’ve added a loop to build out the <code class="language-plaintext highlighter-rouge">$vminfo</code> hash.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code># Iterate through the Nodes, buildig out $vmlist & $vminfo
for( [int]$Node = 1; $Node -le $NodeCount; $Node++)
{
# list of machines to make - hostname will be set to this unless overridden
$vmlist += "coreos$node"
# Determine our IP
$thisIP = ($IPAddressStart.Split(".")[0,1,2] -join ".")+"."+$([int]($IPAddressStart.Split(".")[3])+$Node)
# Add hashmap of machine specific properties
$vminfo["coreos$node"] = @{'interface.0.ip.0.address'="$thisIP/$Cidr"}
}
</code></pre></div></div>
<p>Converting cloud-config to base64 encoded data</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kr">if</span><span class="w"> </span><span class="p">(</span><span class="n">Test-Path</span><span class="w"> </span><span class="o">.</span><span class="nx">\cloud-config.yml</span><span class="p">)</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="c"># pull in the cloud config content</span><span class="w">
</span><span class="nv">$RawCloudConfig</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Get-Content</span><span class="w"> </span><span class="s2">"cloud-config.yml"</span><span class="w"> </span><span class="nt">-raw</span><span class="w">
</span><span class="c"># Encode as UTf8 in bytes</span><span class="w">
</span><span class="nv">$bytes</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">[</span><span class="n">System.Text.Encoding</span><span class="p">]::</span><span class="n">UTF8.GetBytes</span><span class="p">(</span><span class="nv">$RawCloudConfig</span><span class="p">)</span><span class="w">
</span><span class="c"># Add to the properties hash the coreos config data & specify the encoding</span><span class="w">
</span><span class="nv">$gProps</span><span class="p">[</span><span class="s1">'coreos.config.data'</span><span class="p">]</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">[</span><span class="n">System.Convert</span><span class="p">]::</span><span class="n">ToBase64String</span><span class="p">(</span><span class="nv">$bytes</span><span class="p">)</span><span class="w">
</span><span class="nv">$gProps</span><span class="p">[</span><span class="s1">'coreos.config.data.encoding'</span><span class="p">]</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'base64'</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="kr">else</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="kr">Throw</span><span class="w"> </span><span class="s2">"No cloud-config.yml found. Please create and add to this folder"</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>The actual injection into vmx</p>
<div class="language-powershell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># get the file and strip out any existing guestinfo</span><span class="w">
</span><span class="nv">$vmx</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">((</span><span class="n">Get-Content</span><span class="w"> </span><span class="nv">$vmxLocal</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">Select-String</span><span class="w"> </span><span class="nt">-Pattern</span><span class="w"> </span><span class="nx">guestinfo</span><span class="w"> </span><span class="o">-NotMatch</span><span class="p">)</span><span class="w"> </span><span class="o">-join</span><span class="w"> </span><span class="s2">"</span><span class="se">`n</span><span class="s2">"</span><span class="p">)</span><span class="o">.</span><span class="nf">Trim</span><span class="p">()</span><span class="w">
</span><span class="nv">$vmx</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"</span><span class="si">$(</span><span class="nv">$vmx</span><span class="si">)</span><span class="se">`n</span><span class="s2">"</span><span class="w">
</span><span class="c"># build the property bag</span><span class="w">
</span><span class="nv">$props</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$gProps</span><span class="w">
</span><span class="nv">$props</span><span class="p">[</span><span class="s1">'hostname'</span><span class="p">]</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$vmname</span><span class="w">
</span><span class="nv">$vminfo</span><span class="p">[</span><span class="nv">$vmname</span><span class="p">]</span><span class="o">.</span><span class="nf">Keys</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">ForEach-Object</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nv">$props</span><span class="p">[</span><span class="bp">$_</span><span class="p">]</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$vminfo</span><span class="p">[</span><span class="nv">$vmname</span><span class="p">][</span><span class="bp">$_</span><span class="p">]</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="c"># add to the VMX</span><span class="w">
</span><span class="nv">$props</span><span class="o">.</span><span class="nf">Keys</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">ForEach-Object</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nv">$vmx</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"</span><span class="si">$(</span><span class="nv">$vmx</span><span class="si">)</span><span class="s2">guestinfo.</span><span class="si">$(</span><span class="bp">$_</span><span class="si">)</span><span class="s2"> = ""</span><span class="si">$(</span><span class="nv">$props</span><span class="p">[</span><span class="bp">$_</span><span class="p">]</span><span class="si">)</span><span class="s2">""</span><span class="se">`n</span><span class="s2">"</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="c"># write out the VMX</span><span class="w">
</span><span class="nv">$vmx</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">Out-File</span><span class="w"> </span><span class="nv">$vmxLocal</span><span class="w"> </span><span class="nt">-Encoding</span><span class="w"> </span><span class="nx">ascii</span><span class="w">
</span></code></pre></div></div>Brian MarshI’ve worked with CoresOs in the past, but thought that two years was enough of a gap before I attempted to deal with it again.New beginnings2016-09-27T00:00:00+00:002016-09-27T00:00:00+00:00https://devmeetsops.com//site%20news/new-beginnings<p>I’ve decided to drop ghost and switch to jekyll/github pages to reduce my monthly expenses with my previous cloud provider. I looked into continuing with something like <a href="https://github.com/paladini/ghost-on-github-pages">Ghost on Gitub Pages</a>, but upgrades seem clunky and there are too many links that end up redirecting to <code class="language-plaintext highlighter-rouge">localhost</code>.</p>
<p>Quick write up of this blog for future me:</p>
<ul>
<li>This hosted on <a href="https://pages.github.com/">github pages</a></li>
<li>With <a href="https://sheharyar.me/blog/free-ssl-for-github-pages-with-custom-domains/">SSL by Cloudflare</a></li>
<li>Running the <a href="https://mademistakes.com/work/minimal-mistakes-jekyll-theme/">Minimal Mistakes Theme</a></li>
</ul>Brian MarshI’ve decided to drop ghost and switch to jekyll/github pages to reduce my monthly expenses with my previous cloud provider. I looked into continuing with something like Ghost on Gitub Pages, but upgrades seem clunky and there are too many links that end up redirecting to localhost.Domoticz and home automation2016-04-06T00:00:00+00:002016-04-06T00:00:00+00:00https://devmeetsops.com//automation/domoticz-home-automation<p>When I moved into my new townhome, one of the first purchases I made was a 2nd generation Nest thermostat. It’s been working fine - the last software update improved auto-away to the point where it’s now useful - but the sole thermometer is in the entryway on the first floor. This causes issues as the majority of the living space is either the bedrooms upstairs, or the living/tv room in the basement. As of now, Nest doesn’t offer remote temperature nodes, so I have been looking at what other home automation tools I could use to make my smart thermostat even smarter.</p>
<p>My goal is to have remote temperature sensors that can occasionally (perhaps based on proximity/noise/other sensors) override the Nest and increase/decrease the temperature until the occupied portion of my house reaches the desired temperature.</p>
<h1 id="domoticz">Domoticz</h1>
<p>I stumbled upon <a href="https://domoticz.com/">Domoticz</a> while researching open-source/inexpensive home automation software that can manage a Nest thermostat. It has an impressive list of supported hardware:</p>
<ul>
<li>RFXCOM Transceiver</li>
<li>Z-Wave</li>
<li>P1 Smart Meter</li>
<li>1-Wire</li>
<li>Many more</li>
</ul>
<p>Nest support is in its infancy - but I can at least see the current temperature/humidity readings, set Away Mode, and change the target temperature. The software itself has a decent web-based interface (that translates well to mobile).</p>
<p><img src="/images/posts/domoticz1.png" alt="Main Page" /></p>
<p>Historical views are available for each sensor - something I’m interested in using to track the guest bedroom’s ambient temperature this summer.</p>
<p><img src="/images/posts/domoticz2.png" alt="Historical View" /></p>
<p>With the latest update, adjusting the thermostat finally works!
<img src="/images/posts/domoticz3.png" alt="Changing the Temperature" /></p>
<p>Other bonuses: Domoticz runs well on a Raspberry Pi (and even can directly utilize the GPIO ports), has a well documented <a href="https://www.domoticz.com/wiki/Domoticz_API/JSON_URL's">API</a>/<a href="https://www.domoticz.com/wiki/Domoticz_Wiki_Manual">extensive wiki</a>, and active <a href="http://www.domoticz.com/forum/">community forum</a>. I opted to use a spare Raspberry Pi 2 and the provided image.</p>
<h1 id="remote-temperature-sensor">Remote Temperature Sensor</h1>
<p>To address the issue of remote temperature sensors, I picked up a few <a href="http://smile.amazon.com/Sunkee-Digital-Temperature-Humidity-Replace/dp/B00CW82DHG/">DHT22</a> sensors - one for the local Domoticz Raspberry Pi which resides in the basement, one for the Raspberry Pi Zero that will live up in the guest bedroom on the second floor.</p>
<p>These little digital temperature/humidity sensors are pretty accurate, and easy to use with just three pins + 10k resistor.</p>
<p>Wiring is relatively straightforward: 3.3v, ground, and gpio with a 10k resistor between ground & data. There is a decent write-up on the physical wiring on [privateeyepi][9].</p>
<h2 id="software-configuration">Software Configuration</h2>
<p>Two things need to be configured in order for the temperature sensor to work: adding the hardware switch/sensor in the Domoticz software and create the underlying script to pull data from the sensor.</p>
<h2 id="domoticz-settings">Domoticz Settings</h2>
<ul>
<li>Create a dummy switch (Setup > Hardware) <img src="/images/posts/domoticz4.png" alt="Dummy Switch" /></li>
<li>Create the associated sensor hardware when prompted - we want a temperature + humidity as sensor type (note the resulting IDX, this will be used in the script below).</li>
</ul>
<h2 id="sensor-script">Sensor Script</h2>
<p>For the script, do the following:</p>
<p>Installing Adafruit and WiringPi is simple enough. The bash script has a few sections that need customization.</p>
<h3 id="connection-details">Connection Details</h3>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> # Domoticz server
SERVER="localhost:8080"
</code></pre></div></div>
<p>The Domoticz server needs to be adjusted (for my guest bedroom I put in the FQDN or IP address):</p>
<h3 id="hardware-details">Hardware Details</h3>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> # IDX DHT
# The number of the IDX in the list of peripherals
DHTIDX="4"
#DHTPIN
# THE GPIO or connects DHT11
DHTPIN="17"
</code></pre></div></div>
<p>Domoticz assigns an index to each device, that index needs to be updated for <code class="language-plaintext highlighter-rouge">DHTIDX</code>. The GPIO in which the dataport is connected needs to be updated for <code class="language-plaintext highlighter-rouge">DHTPIN</code>.</p>
<p>Running the script manually should both update the Domoticz sensor and output to console the resolved humidity and temperature readings. Add this to cron and you should be good to go!</p>
<h1 id="other-things">Other things</h1>
<p>The “comfort” level is arbitrary at this point based exclusively on relative humidity - not dew point. as a result, you can have a situation like this:
<img src="/images/posts/domoticz.png" alt="Is it dry or comfortable?" /></p>
<p>How is 34% humidity comfortable, but 37% is dry? Also note that dew point is within ~1.5 degrees. This script could probably do with some calculations of dew point and humidity ranges to better calculate the comfort level.</p>Brian MarshWhen I moved into my new townhome, one of the first purchases I made was a 2nd generation Nest thermostat. It’s been working fine - the last software update improved auto-away to the point where it’s now useful - but the sole thermometer is in the entryway on the first floor. This causes issues as the majority of the living space is either the bedrooms upstairs, or the living/tv room in the basement. As of now, Nest doesn’t offer remote temperature nodes, so I have been looking at what other home automation tools I could use to make my smart thermostat even smarter.Moving, home lab status, and PowerShell reporting2015-09-08T00:00:00+00:002015-09-08T00:00:00+00:00https://devmeetsops.com//home%20lab/moving-home-lab-status<p>Wow, so a lot to cover today. I recently moved out of my parents basement and into my own townhome with the wife and dog! Unfortunately, that means that we’re back to paying 100% of the utilities and I’m giving some strong consideration to changing my home lab from an always on to a only-when-I-need-it. I would leave my Synology on for storage/backups, but turn off the HP DL360s and possibly the Avaya switch. New hardware would need to be purchased for routing (currently pfSense is virtual).</p>
<p>Between iLO and Wake On LAN, I think I can come up with something that will enable me to do a single button <strong>BAM</strong> home lab workflow/automation. More on that later (I swear).</p>
<h1 id="me-report-pretty-one-day">Me Report Pretty One Day</h1>
<p>The bulk of this post will be surrounding “pretty” reporting. I rely heavily upon PowerShell for reports - everything from VMware snapshots to UCS Service Profile usage. PowerShell is great for session based reporting (hey Brian, what’s the biggest VM storage footprint right now?) but it has a pretty sizable weakness for exporting comprehensive reports in a useful, accessible manner.</p>
<h2 id="export-csv">Export-CSV</h2>
<p>CSVs were the first (and most effortless) way of getting data out of PS, but frankly, they’re kind of two dimensional. Let’s get all the VMs in my homelab, and report on GuestOS/CPU/memory configuration.</p>
<figure class="highlight"><pre><code class="language-powershell" data-lang="powershell"><span class="n">Connect-VIServer</span><span class="w"> </span><span class="nx">vcenter.pezlab.local</span><span class="w">
</span><span class="nv">$vms</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Get-VM</span><span class="w">
</span><span class="nv">$report</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nv">$vms</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">select</span><span class="w"> </span><span class="nx">Name</span><span class="p">,</span><span class="w"> </span><span class="nx">GuestId</span><span class="p">,</span><span class="w"> </span><span class="nx">NumCPU</span><span class="p">,</span><span class="w"> </span><span class="nx">MemoryGB</span><span class="w">
</span><span class="nv">$report</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">Export-Csv</span><span class="w"> </span><span class="nx">c:\temp\report.csv</span><span class="w"> </span><span class="nt">-NoTypeInformation</span></code></pre></figure>
<p><img src="/images/posts/CSV-1.png" alt="Sample CSV" /></p>
<p>CSVs don’t offer any formatting, no auto-sizing of cells, sorting is a major PITA… and if you want to collect multiple reports in a single file? Not possible (goodbye single “State of VMware” file containing multiple reports on multiple different vCenters).</p>
<h2 id="excel">Excel</h2>
<p>The next iteration in external reporting is Excel - it will handle multiple worksheets (e.g. reports) in a single file, formatting is not only available it can be quite powerful (think heat maps), and Excel can do fancy graphs! However getting data from PowerShell into Excel isn’t as easy as you might hope. <a href="http://blogs.technet.com/b/heyscriptingguy/archive/2014/01/10/powershell-and-excel-fast-safe-and-reliable.aspx">Hey Scripting Guy</a> has a great article on how to utilize the Excel COM object to directly write data into an xlsx file - but this is both clunky and time consuming (and requires Excel be installed on the system running the script).</p>
<p>Most recently, I’ve been using <a href="https://github.com/dfinke/ImportExcel">ImportExcel</a>. This PowerShell module leverages the <a href="http://epplus.codeplex.com/">EPPlus</a> library to directly create xlsx files without the need for Excel to be installed.</p>
<p>Let’s try the same bit of code but exporting the <code class="language-plaintext highlighter-rouge">$report</code> variable with <code class="language-plaintext highlighter-rouge">Export-Excel</code>:</p>
<p><code class="language-plaintext highlighter-rouge">$report | Export-Excel -Path c:\tempreport.xlsx -WorksheetName "VM Report" -BoldTopRow -AutoSize</code></p>
<p><img src="/images/posts/xlsx.png" alt="Excel Output" /></p>
<p>This module (and others like it) work quite well, but Excel in general has some downsides:</p>
<ul>
<li>Requires Excel for viewing files (pretty much everyone has Excel, but if you’re looking to view a report from a Server the chances of Excel being installed is slim).</li>
<li>Historical report comparison is difficult.</li>
<li>Only one person can edit an Excel document at a time (and it’s very easy to edit/change report data).</li>
<li>Multiple Excel documents cannot easily be compared (e.g. let’s see how a UCS blade relates to a VMware host and what VMs are residing on that host’s cluster).</li>
</ul>
<h2 id="sql-visual-studio-and-gridview">SQL, Visual Studio, and Gridview</h2>
<p>One potential solution for each of those downsides to Excel is a web app with database back-end. Complex, historical reports can be pulled via database queries and displayed for multiple concurrent users while making it more difficult for report data to be tampered with. The only requirement for viewing a web page is a browser - something that nearly every modern operating system includes.</p>
<p>For this self-hosted PoC, I installed <a href="https://www.microsoft.com/en-us/download/details.aspx?id=42299">SQL Server 2014 Express</a> and created a <strong>vstest</strong> user, a <strong>testweb</strong> database, and a simple webpage using the Visual Studio template for ASP.NET Web Forms. This time around, I leveraged the <a href="https://gallery.technet.microsoft.com/scriptcenter/Write-object-to-database-7be1d3c5">Write-ObjectToSQL</a> (I’m not completely sold on this module yet, but it was a quick ‘n dirty way to get my data into a table - and it worked so, woo?).</p>
<p><code class="language-plaintext highlighter-rouge">$report | Write-ObjectToSQL -Server localhostSQLEXPRESS -Database Testweb -TableName vms</code></p>
<p><img src="/images/posts/sql.png" alt="Data in SQL" /></p>
<p>I edited the default Visual Studio template content and inserted an <code class="language-plaintext highlighter-rouge">asp:GridView</code> tag, then switched to design view to finish configuring the SQL endpoint.</p>
<p><img src="/images/posts/vsGridView.png" alt="Visual Studio GridView" /></p>
<p>After configuring my data source, I was able to change the headers into something more reasonable, change the column order, and add some general information.</p>
<figure class="highlight"><pre><code class="language-html" data-lang="html"><span class="nt"><</span><span class="err">%@</span> <span class="na">Page</span> <span class="na">Title=</span><span class="s">"Home Page"</span> <span class="na">Language=</span><span class="s">"C#"</span> <span class="na">MasterPageFile=</span><span class="s">"~/Site.Master"</span> <span class="na">AutoEventWireup=</span><span class="s">"true"</span> <span class="na">CodeFile=</span><span class="s">"Default.aspx.cs"</span> <span class="na">Inherits=</span><span class="s">"_Default"</span> <span class="err">%</span><span class="nt">></span>
<span class="nt"><asp:Content</span> <span class="na">ID=</span><span class="s">"BodyContent"</span> <span class="na">ContentPlaceHolderID=</span><span class="s">"MainContent"</span> <span class="na">runat=</span><span class="s">"server"</span><span class="nt">></span>
<span class="nt"><div</span> <span class="na">class=</span><span class="s">"jumbotron"</span><span class="nt">></span>
<span class="nt"><h1></span>VMware Home Lab!<span class="nt"></h1></span>
<span class="nt"><p</span> <span class="na">class=</span><span class="s">"lead"</span><span class="nt">></span>Behold the wonder that is Visual Studio 2015 Community Ed., SQL 2014 Express, and VMware!<span class="nt"></p></span>
<span class="nt"><p></span>Data pulled from PowerShell, dumped to SQL, and then pulled using Visual Studio 2015 <span class="err">&</span> GridView<span class="nt"></p></span>
<span class="nt"></div></span>
<span class="nt"><asp:GridView</span> <span class="na">Id=</span><span class="s">"vms"</span> <span class="na">runat=</span><span class="s">"server"</span> <span class="na">AutoGenerateColumns=</span><span class="s">"False"</span> <span class="na">DataSourceID=</span><span class="s">"SqlDataSource1"</span> <span class="na">AllowSorting=</span><span class="s">"True"</span><span class="nt">></span>
<span class="nt"><Columns></span>
<span class="nt"><asp:BoundField</span> <span class="na">DataField=</span><span class="s">"Name"</span> <span class="na">HeaderText=</span><span class="s">"Name"</span> <span class="na">SortExpression=</span><span class="s">"Name"</span> <span class="nt">/></span>
<span class="nt"><asp:BoundField</span> <span class="na">DataField=</span><span class="s">"GuestId"</span> <span class="na">HeaderText=</span><span class="s">"Guest ID"</span> <span class="na">SortExpression=</span><span class="s">"GuestId"</span> <span class="nt">/></span>
<span class="nt"><asp:BoundField</span> <span class="na">DataField=</span><span class="s">"NumCpu"</span> <span class="na">HeaderText=</span><span class="s">"vCpus"</span> <span class="na">SortExpression=</span><span class="s">"NumCpu"</span> <span class="nt">/></span>
<span class="nt"><asp:BoundField</span> <span class="na">DataField=</span><span class="s">"MemoryGB"</span> <span class="na">HeaderText=</span><span class="s">"Memory (GB)"</span> <span class="na">SortExpression=</span><span class="s">"MemoryGB"</span> <span class="nt">/></span>
<span class="nt"></Columns></span>
<span class="nt"></asp:GridView></span>
<span class="nt"><asp:SqlDataSource</span> <span class="na">ID=</span><span class="s">"SqlDataSource1"</span> <span class="na">runat=</span><span class="s">"server"</span> <span class="na">ConnectionString=</span><span class="s">"<%$ ConnectionStrings:TestwebConnectionString %>"</span> <span class="na">SelectCommand=</span><span class="s">"SELECT [GuestId], [MemoryGB], [Name], [NumCpu] FROM [vms]"</span><span class="nt">></asp:SqlDataSource></span>
<span class="nt"></asp:Content></span></code></pre></figure>
<p>The final result can be seen here in a screengrab:</p>
<p><img src="/images/posts/webpage.png" alt="VMware Report" /></p>
<h2 id="whats-next">What’s Next</h2>
<p>Although this is working in my <em>very</em> limited PoC, it is working! My next steps will be to figure out:</p>
<ul>
<li>How should the final reporting page work? (Do I want individual URLs for UCS & VMware, or one reporting URL to rule them all?)</li>
<li>How will I handle stored credentials for my automated PowerShell scripts to talk to SQL?</li>
<li>How will this translate to a more production-esque deployment?</li>
</ul>Brian MarshWow, so a lot to cover today. I recently moved out of my parents basement and into my own townhome with the wife and dog! Unfortunately, that means that we’re back to paying 100% of the utilities and I’m giving some strong consideration to changing my home lab from an always on to a only-when-I-need-it. I would leave my Synology on for storage/backups, but turn off the HP DL360s and possibly the Avaya switch. New hardware would need to be purchased for routing (currently pfSense is virtual).Adventures with Raspberry Pi2015-07-27T00:00:00+00:002015-07-27T00:00:00+00:00https://devmeetsops.com//automation/adventures-with-raspberry-pi<p><a href="http://smile.amazon.com">Amazon</a> recently celebrated their Prime Day, offering <strong><em>amazing</em></strong> deals - most of which amounted to only a few dollars off their already reduced prices. However, there was one gem that I managed to pick up: the <a href="http://smile.amazon.com/Raspberry-components--Raspberry-Guide--Edimax-Adapter--8GB-Adapter--Case--Power/dp/B00PWYK2V6/">Raspberry Pi 2 Ultimate Starter Kit</a> for only $55! I’ve been wanting to get a rpi2 for a while (ever since I saw the updated specs and after hearing that it supports embedded Windows 10). I have the first project idea in mind, but quickly found that I’ll need to purchase a few more rpi2s for other projects. While I’m waiting for the first project’s parts to arrive, I thought I’d throw together the comprehensive (as of now) outline for future projects.</p>
<h1 id="wifibluetooth-audio-streaming-server">WiFi/Bluetooth audio streaming server</h1>
<p>By cramming the rpi2 into an old stereo, I can upgrade an old piece of hardware and give it new life. There are a few products out there that can do streaming (<a href="https://volumio.org/">Volumio</a>, <a href="http://www.runeaudio.com/">RuneAudio</a>, etc) however, none of them natively support my music source of choice, Google Music - All Access. In theory <a href="http://www.pimusicbox.com/">Pi MusicBox</a> does support Google Music (as well as SoundCloud, Spotify, and a whole host of other options). It doesn’t natively appear to support bluetooth (which isn’t <em>as</em> critical if it can do Google Music), but I may be able to combine it with <a href="http://www.instructables.com/id/Turn-your-Raspberry-Pi-into-a-Portable-Bluetooth-A/">this instructable</a> to do both Wifi and Bluetooth.</p>
<p>I’ve purchased (for a mere $7) a <a href="http://i.imgur.com/tSgN2mp.jpg">Sanyo boombox</a> (Woo! Potato Quality Photo!), and after extensive hacked together testing, I realized the rpi2 has crap audio output. It needs an amp. And a DAC. Enter the <a href="https://www.hifiberry.com/ampplus/">HiFiBerry AMP+</a> - or rather, enter the AMP+ once it arrives from Switzerland. It requires a 12V power supply, and luckily the output from the stock Sanyo psu is right around 11.87V! I need to check the amp pull, but even if things don’t work out, I can always hit up the trusty electronic store to grab a new PSU. Expect some further posts once the AMP+ arrives!</p>
<h1 id="smart-power-switch">Smart Power Switch</h1>
<p>My cable modem keeps crapping out, removing my remote administration (and IRC capabilities). THIS WILL NOT STAND. Unfortunately, the only real fix appears to be unplugging the modem, waiting a bit, then plugging it back in (the power supply is probably on its way out after a few years of 24/7 uptime. I <em>could</em> replace the power, but why not add to the Internet of Things? <a href="https://www.youtube.com/watch?v=8cPK8lh6oLI">This youtube video</a> shows how to use a rpi2 to control a power strip for remote power control. Perfect! My goal is to leverage <a href="http://webiopi.trouch.com/">webiopi</a> for the fancy web interface to control the power outlet. (Alternatively this <a href="http://hackaday.com/2015/02/11/wifi-controlled-power-outlets-with-raspberry-pi/">Hackaday project</a> could fit the bill. But wait Brian, how will you power cycle the modem if the internet is down? Well voice in my head, a simple script on my linux jump box should handle that! Continuous pinging to google, when it fails, send the REST API call to power off the outlet, wait 30 seconds, and then power it back on. If stuff still doesn’t work, call it a day (or maybe retry a few times - this requires experimentation). I need further research to see how to properly do this project safely…</p>
<h1 id="smart-thermostat">Smart Thermostat</h1>
<p>The <a href="http://smile.amazon.com/Nest-Learning-Thermostat-2nd-Generation/dp/B009GDHYPQ">Nest Thermostat</a> is cool, but really expensive. I’m thinking that with some effort, thinking, and parts, I can accomplish something similar for around $100-150. <a href="http://imgur.com/gallery/YxElS">There</a> are <a href="https://www.raspberrypi.org/forums/viewtopic.php?f=37&t=95682">several</a> sources <a href="https://lifehacker.com/build-a-web-connected-thermostat-with-a-raspberry-pi-an-1670379446">on</a> the <a href="http://wyattwinters.com/rubustat-the-raspberry-pi-thermostat.html">internet</a> that cover various ways to accomplish this. Some directly tie into the wiring for the thermostat, others seem to just trigger the button presses to adjust temperature. I really like the direction that Jeff took things - multiple room have independent temperature sensors. The rpi2 can take an average and use that to drive the thermostat or alternatively, use only one (e.g. I’m going to bed and want my room to be 74°F - and the AC/Heat runs until that sensor registers the proper temperature). I’m hopefully going to be moving into a rental shortly - depending on how much freedom I have to take things apart, this may end up being purely academic.</p>
<h1 id="murphy-monitoring">Murphy Monitoring</h1>
<p>I have an awesome dog named <a href="https://i.imgur.com/VmLYaQS.png">Murphy</a>. I’ve always wondered how he would do if we left him out instead of crating him when we run errands. Maybe I could turn my rpi2 into a <a href="http://www.averagemanvsraspberrypi.com/2014/09/turn-raspberry-pi-into-cctv-security.html">CCTV Camera</a> and keep an eye on him? There’s probably more options out there, I’ll have to keep looking.</p>
<h1 id="other-fun-stuff">Other fun stuff</h1>
<p>I just found this <a href="http://www.piups.net/">Micro UPS</a> - a battery backup solution for the rpi2. If I’m doing the Smart Thermostat or Murphy Monitoring (trademark pending) I’ll need to have some battery backup in case of power outage - there have been some nasty stories about SD Card corruption when power loss and data writing are involved. I may leverage Arduino to make a home security system (one of the places I’m considering renting has a basement with unprotected windows). Maybe something like <a href="http://www.freetronics.com.au/products/security-sensor-shield">this Arduino security sensor</a> could be cool.</p>Brian MarshAmazon recently celebrated their Prime Day, offering amazing deals - most of which amounted to only a few dollars off their already reduced prices. However, there was one gem that I managed to pick up: the Raspberry Pi 2 Ultimate Starter Kit for only $55! I’ve been wanting to get a rpi2 for a while (ever since I saw the updated specs and after hearing that it supports embedded Windows 10). I have the first project idea in mind, but quickly found that I’ll need to purchase a few more rpi2s for other projects. While I’m waiting for the first project’s parts to arrive, I thought I’d throw together the comprehensive (as of now) outline for future projects.PowerCLI - configuring iSCSI from scratch2015-07-27T00:00:00+00:002015-07-27T00:00:00+00:00https://devmeetsops.com//automation/powercli-configuring-iscsi-from-scratch<p>I threw this together a while ago, it will help configure iSCSI on a VM host using PowerCLI. Note that once you do a find/replace to set your iSCSI network (e.g. 192.168.0.0/24 192.168.1.0/24), all you’ll need to change is the $thisHost, $lastOctet, and $targets values before running.</p>
<p>Requires magic from <a href="http://www.jonathanmedd.net/2013/07/using-powercli-for-iscsi-vmkernel-port-binding.html">Jonathan Medd’s Blog</a> (for the Set-VMHostiSCSIBinding cmdlet).</p>
<figure class="highlight"><pre><code class="language-powershell" data-lang="powershell"><span class="c"># This configures iSCSI from scratch!</span><span class="w">
</span><span class="c"># Set the specific information for this host</span><span class="w">
</span><span class="bp">$this</span><span class="n">Host</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Get-VMHost</span><span class="w"> </span><span class="nx">dc2server03</span><span class="o">*</span><span class="w">
</span><span class="nv">$lastOctet</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="mi">62</span><span class="w">
</span><span class="nv">$iSCSI_addr1</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"192.168.0.</span><span class="nv">$lastOctet</span><span class="s2">"</span><span class="w">
</span><span class="nv">$iSCSI_addr2</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"192.168.1.</span><span class="nv">$lastOctet</span><span class="s2">"</span><span class="w">
</span><span class="c"># Get the iSCSI dvSwitch Port Groups</span><span class="w">
</span><span class="nv">$iSCSI_dvPG1</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Get-VDPortgroup</span><span class="w"> </span><span class="s2">"iSCSI dvPG1"</span><span class="w">
</span><span class="nv">$iSCSI_dvPG2</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Get-VDPortgroup</span><span class="w"> </span><span class="s2">"iSCSI dvPG2"</span><span class="w">
</span><span class="nv">$iSCSI_dvSw</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">Get-VirtualSwitch</span><span class="w"> </span><span class="nt">-name</span><span class="w"> </span><span class="s2">"UCS iSCSI dvSwitch"</span><span class="w">
</span><span class="c"># Create a new host network adapter for this host on both dvPGs</span><span class="w">
</span><span class="n">New-VMHostNetworkAdapter</span><span class="w"> </span><span class="nt">-VMHost</span><span class="w"> </span><span class="bp">$this</span><span class="nx">Host</span><span class="w"> </span><span class="nt">-PortGroup</span><span class="w"> </span><span class="nv">$iSCSI_dvPG1</span><span class="w"> </span><span class="nt">-IP</span><span class="w"> </span><span class="nv">$iSCSI_addr1</span><span class="w"> </span><span class="nt">-SubnetMask</span><span class="w"> </span><span class="nx">255.255.255.0</span><span class="w"> </span><span class="nt">-Mtu</span><span class="w"> </span><span class="nx">9000</span><span class="w"> </span><span class="nt">-VirtualSwitch</span><span class="w"> </span><span class="nv">$iSCSI_dvSw</span><span class="w">
</span><span class="n">New-VMHostNetworkAdapter</span><span class="w"> </span><span class="nt">-VMHost</span><span class="w"> </span><span class="bp">$this</span><span class="nx">Host</span><span class="w"> </span><span class="nt">-PortGroup</span><span class="w"> </span><span class="nv">$iSCSI_dvPG2</span><span class="w"> </span><span class="nt">-IP</span><span class="w"> </span><span class="nv">$iSCSI_addr2</span><span class="w"> </span><span class="nt">-SubnetMask</span><span class="w"> </span><span class="nx">255.255.255.0</span><span class="w"> </span><span class="nt">-Mtu</span><span class="w"> </span><span class="nx">9000</span><span class="w"> </span><span class="nt">-VirtualSwitch</span><span class="w"> </span><span class="nv">$iSCSI_dvSw</span><span class="w">
</span><span class="c"># Setup the iSCSI Binding</span><span class="w">
</span><span class="bp">$this</span><span class="n">Host</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">Set-VMHostiSCSIBinding</span><span class="w"> </span><span class="nt">-HBA</span><span class="w"> </span><span class="nx">vmhba32</span><span class="w"> </span><span class="nt">-VMKernel</span><span class="w"> </span><span class="s2">"vmk1"</span><span class="w">
</span><span class="bp">$this</span><span class="n">Host</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">Set-VMHostiSCSIBinding</span><span class="w"> </span><span class="nt">-HBA</span><span class="w"> </span><span class="nx">vmhba32</span><span class="w"> </span><span class="nt">-VMKernel</span><span class="w"> </span><span class="s2">"vmk2"</span><span class="w">
</span><span class="c"># Configure the iSCSI targets</span><span class="w">
</span><span class="nv">$targets</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"192.168.0.10"</span><span class="p">,</span><span class="s2">"192.168.0.20"</span><span class="p">,</span><span class="s2">"192.168.1.10"</span><span class="p">,</span><span class="s2">"192.168.1.20"</span><span class="w">
</span><span class="nv">$hba</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="bp">$this</span><span class="n">Host</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">Get-VMHostHba</span><span class="w"> </span><span class="nt">-Type</span><span class="w"> </span><span class="nx">iScsi</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">Where</span><span class="w"> </span><span class="p">{</span><span class="bp">$_</span><span class="o">.</span><span class="nf">Model</span><span class="w"> </span><span class="o">-eq</span><span class="w"> </span><span class="s2">"iSCSI Software Adapter"</span><span class="p">}</span><span class="w">
</span><span class="kr">foreach</span><span class="p">(</span><span class="nv">$target</span><span class="w"> </span><span class="kr">in</span><span class="w"> </span><span class="nv">$targets</span><span class="p">){</span><span class="w">
</span><span class="kr">if</span><span class="p">(</span><span class="n">Get-IScsiHbaTarget</span><span class="w"> </span><span class="nt">-IScsiHba</span><span class="w"> </span><span class="nv">$hba</span><span class="w"> </span><span class="nt">-Type</span><span class="w"> </span><span class="nx">Send</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">Where</span><span class="w"> </span><span class="p">{</span><span class="bp">$_</span><span class="o">.</span><span class="nf">Address</span><span class="w"> </span><span class="o">-cmatch</span><span class="w"> </span><span class="nv">$target</span><span class="p">}){</span><span class="w">
</span><span class="n">Write-Host</span><span class="w"> </span><span class="s2">"The target </span><span class="nv">$target</span><span class="s2"> does exist on </span><span class="bp">$this</span><span class="s2">Host"</span><span class="w"> </span><span class="nt">-ForegroundColor</span><span class="w"> </span><span class="nx">Green</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="kr">else</span><span class="p">{</span><span class="w">
</span><span class="n">Write-Host</span><span class="w"> </span><span class="s2">"The target </span><span class="nv">$target</span><span class="s2"> doesn't exist on </span><span class="bp">$this</span><span class="s2">Host"</span><span class="w"> </span><span class="nt">-ForegroundColor</span><span class="w"> </span><span class="nx">Red</span><span class="w">
</span><span class="n">Write-Host</span><span class="w"> </span><span class="s2">"Creating </span><span class="nv">$target</span><span class="s2">"</span><span class="w"> </span><span class="nt">-ForegroundColor</span><span class="w"> </span><span class="nx">Yellow</span><span class="w">
</span><span class="n">New-IScsiHbaTarget</span><span class="w"> </span><span class="nt">-IScsiHba</span><span class="w"> </span><span class="nv">$hba</span><span class="w"> </span><span class="nt">-Address</span><span class="w"> </span><span class="nv">$target</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">Out-Null</span><span class="w">
</span><span class="nx">Write-Host</span><span class="w"> </span><span class="s2">"done..."</span><span class="w"> </span><span class="nt">-ForegroundColor</span><span class="w"> </span><span class="nx">Green</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="c"># Rescan to find iSCSI datastores</span><span class="w">
</span><span class="bp">$this</span><span class="n">Host</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">Get-VMHostStorage</span><span class="w"> </span><span class="nt">-RescanAllHba</span><span class="w">
</span><span class="c"># Just a little clean up, rename any default local datastores</span><span class="w">
</span><span class="bp">$this</span><span class="n">Host</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">get-datastore</span><span class="w"> </span><span class="nx">datastore</span><span class="o">*</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">set-datastore</span><span class="w"> </span><span class="nt">-name</span><span class="w"> </span><span class="s2">"</span><span class="si">$(</span><span class="bp">$this</span><span class="n">Host.name.Substring</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span><span class="bp">$this</span><span class="n">Host.Name.IndexOf</span><span class="p">(</span><span class="s2">"."</span><span class="si">)</span><span class="s2">))_boot"</span></code></pre></figure>Brian MarshI threw this together a while ago, it will help configure iSCSI on a VM host using PowerCLI. Note that once you do a find/replace to set your iSCSI network (e.g. 192.168.0.0/24 192.168.1.0/24), all you’ll need to change is the $thisHost, $lastOctet, and $targets values before running.