Windows Server Backup

Occasionally a client does not want to pay for backup software, like StorageCraft, AppAssure, Veeam, whatever gets their 50 employees back up quickly is not worth $1k to management.

In that case, the cheapest possible solution we use is built-in Windows Server Backup. God-forbid, you have to use something like Symantec Backup Exec, which is really only designed for Tape-Drives — not USB HDDs (again, think cheap).

Windows Server Backup by default, when you build your schedule will only use 1 USBHDD. There is no GUI option to add other USBHDDs to a pool of drives.

 

Adding USBHDDs to a Windows Server Backup Pool

You’ll set up your first USBHDD through the GUI via the scheduler tool. Pick a backup time frame and you’re done. To add new USBHDDs to the pool is a fairly simple process.

  1. Connect the new USBHDD (even if it’s already formatted or linked to another backup server).
  2. ::Local Command Prompt (Run as Administrator)
    ::Pull the Disk GUIDs
    wbadmin get disks
    
    :: The GUID includes a long string inside of brackets { }.
    
    ::Copy-Paste the Disk GUID into the following blank, replacing what is in the brackets
    WBADMIN ENABLE BACKUP -addtarget:{12345678-0000-0000-0000-000000000000}
    
    ::Y - Yes, Y-Yes, wait, eject the drive, and switch it out. Drive automatically adds itself into the scheduling pool.
  3. Add the drives one by one and get a USBHDD pool created and you’re done. If any of the USBHDDs is connected, the next scheduled backup will use that disk.

Quickbooks Error H505 – Multi-User Hosting

Quickbooks Error H505

Oh no! My user is trying to open a Quickbooks workbook stored on a shared drive and gets this awful message: Quickbooks Error code H505, “this company file is on another computer, and QuickBooks needs some help connecting.” This is the generic client error for “cannot connect to host”. Directly tied in to the QuickboksDBXX service not running on the hosting server.

 

There are two parts to this fix, and possibly a third.

  1. Running the Quickbooks Component Repair Tool – Effectively fixes .NET and DLL registrations within Quickbooks. Run as Administrator.
    1. https://intuitcorp.quickbase.com/up/bhpb3kw5p/g/rbw/eg/va/QBComponentToolv3.exe
  2. Changing the QuickbooksDBXX service on the hosting server to Run As System, rather than .\QBDataServiceUser26
  3. Occasionally after the first two fixes, you still get an error, -6175,0, reinstall Quickbooks and reconfigure the service to run as System.
  4. Check the firewall on the host server, quick test is to temporarily disable Windows firewall and give it a shot. If it is the firewall, forward the ports:

quickbooks-h505-another-computer

The fix is actually quite simple, head over to the Quickbooks Download Page and download the matching version of your Quickbooks. If you don’t know the info, open the app on a workstation and hit F2 (screencap example)

Quickbooks-Product-Information-2

Quickbooks Product Download Link

https://community.intuit.com/articles/1200542-download-quickbooks-products

Run the installer on the machine hosting the Quickbooks Workbook files, Custom and Network Options > I will be storing our company file here so it can be shared over our network.

However, the Intuit coders make terrible software, and though the QuickbooksDB26 (2016) service is created, you know, the one allowing users to connect to the sQuickbooks-service-not-runningerver with multi-user access…. it doesn’t start… It is set to manual, and attempting to start it provides the worthless message: “The QuickbooksDB26 service started and then stopped”. However, it should be always running in order for users to use QBWs for the matching version. So get this, you *may* have to reboot your server for the services to register correctly, possibly during production hours if it’s urgent from management, so their terrible service can register itself in a way that it functions.

Believe it or not, Quickbooks 2014 and 2015 will actually automatically restart the server, no prompt, no option to hold off, just *poof*, off goes your server. At least 2016 doesn’t do anything (though it doesn’t even let you know its necessary).

 

Don’t forget to go to services.msc > Properties > Startup Type: Automatically. Awesome… At the end of the day, it either works and you are a hero, or it took too long and someone is grumpy, either way, you got the job done!

Quickbooks is trying to access the company file but the database server is not responding – 6175,0

Quickbooks-6175-database-server-not-responding

Oh it gets better. Despite installing the Quickbooks service, rebooting the server, new error! – -6175,0. Basically means the QuickbooksDB26 service is not running, and you can’t start it either.

When trying to start the service, you’ll see: “The QuickbooksDB26 service on local computer started and then stopped”. Despite there being no helpful logs in the event viewer, the problem in my case was a logon issue.

QuickbooksDB26-logon

Run (Windows+R) > Services.msc > Right-Click QuickbooksDB26 > Properties > Log On Tab >

Change “This Account” from .\QBDataServiceUser26 (Local User), to “Local System Account”.

Right-click > Start….

 

 

Wow, finally, my users can work.

 

PHP Server Monitoring Board – Ubuntu 14.04 – AppAssure Installation Missing Dependencies

PHP Monitoring Board w/ AppAssure Backups

We have a pretty slick monitoring board running 24/7 as part of a NOC in our office, the elegant and simple PHP Server Monitor. It’s rock solid, monitors about 250 servers and 100 internet connections, within 1 minute we know if something is down. Really useful because anything down goes into the upper left corner of the screen, when you’ve got 350 devices they would be teeny-tiny unreadable boxes on the TV. It has a update-timer (offline for 38 minutes), and a ping monitor (0.05s) since the last check. Since I set the crontab script to query all devices once per minute, we get minutely updates. A laptop set in Chrome kiosk mode also refreshes the page every 10 seconds for display on the TV

NOC on in-office TV

php-server-monitor-tv-2

Nothing like making a phone call to a client before they even know they are down.

I realized that this VM isn’t backed up, yikes! Off to install the latest AppAssure agent (as of 5.X Linux 14.X is supported), only to find an awful error: “Missing dependencies “linux-libc-dev”. Well ‘lo and behold I had to run something as simple as “apt-get update”, then: the commands worked:

::Make AppAssure agent executable, install
cd /
cd /home/username
wget "http://link.from.appassure.licenseportal.com/Downloads"
chmod +x
./appassure-installer_ubuntu_amd64_5.4.3.106.sh
y (port 8006 + reboot when done)

 

::Per Minute Syncing on PHP Server Monitor
username@PHPMONITOR:~$ sudo -i
[sudo] password for username:
root@PHPMONITOR:~# crontab -e

::Code for minutely updates
*/1 * * * * /usr/bin/php /var/www/html/cron/status.cron.php

 

php-server-mon-appassure-chmod-x

Protect the machine under AppAssure using your hostname/IP, port (8006 by default), username, and password, set a schedule, enjoy super-awesome backups.

RDP Listening Port – Sonicwall NAT Translation or Registry Change

Customize RDP Listening Port

Quite a few customers want to RDP to their local workstation from home. Opening RDP to the public internet can be a massive security risk, but in practice, it’s very useful and “secure enough” as long as you stay of TCP 3389 which botnets tend to brute force once they find it open. (There’s no security like obfuscation right….),

 

There are two ways to pull off a RDP connection on a different port. NAT Translation, and Registry Edit.

NAT translation leaves the target computer listening on Remote Desktop via the stock TCP 3389, but uses the router to translate say, TCP 4000 (Public) –> TCP 3389 (Internal).

Registry edit involves changing the port that Remote Desktop Services listens on, and uses a straight Port Forward (TCP 4005 –> TCP 4005).

You can even mix and match if you really wanted, but K.I.S.S. (Keep it simple stupid) if you can.

 

 

NAT Translation

I prefer NAT translation whenever possible, simpler to modify, keeps workstations stock. Your device will need a static IP or DHCP reservation, like any port-forward would.

Pictures attached below of NAT Translation for a Sonicwall.

Custom Service > RDP-4000 (TCP 4000)

Public Server Wizard -> X.X.X.X (Public) -> 192.168.0.X (Private)

Network > NAT Policies > Add

::::Sonicwall NAT Policy Port-Translation / Redirect
::::Original Source:Any
::::Translated Source:Original
::::Original Destination:Server Public (or Primary WAN IP if you are using the stock network interface of X1)
::::Translated Destination:Server Private
::::Original Service:RDP-4000
::::Translated Service:Terminal Services TCP (3389)
:: - If you choose just "Terminal Services" rather than "Terminal Services TCP", it will fail with error "Unknown Service Class", because that is a group, rather than a single service/port.

Sonicwall-NAT-Translate2

Confirm if the machine is listening on that port for RDP connections.

::Confirm that the port is being listened on.
netstat -ano | find "3389"

rdp-listening2

Port-Query-4002

Registry Edit

Run (Windows+R) > regedit.exe >

::HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Terminal Server\WinStations\RDP-Tcp\PortNumber

Change to your desired port (4001 rather than default 3389)

Or through (Administrative!) Command Line (CLI) and auto-restart the Remote Desktop Services, so you don’t have to reboot to take your change live. No prompts either with the /y quiet switch.

:: Commandline to change the RDP Listening Port
reg add "HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Terminal Server\WinStations\RDP-Tcp" /v PortNumber /t REG_DWORD /d 4001 /f
:: No Reboot Required - Restart Remote Desktop Services to listen with the new port
net stop TermService /y && net start TermService /y

:: Add a firewall rule to make it possible to connect in
netsh advfirewall firewall add rule name="Open RDP 4001" dir=in action=allow protocol=TCP localport=4001

Confirm it’s listening on the new port, awwwww yeah!

netstat-2

 

Hope that helps, leave a post if you want, always glad to hear from new friends 🙂

Microsoft Exchange – Name on the Security Certificate is Invalid or Does Not Match

Exchange – Name on the Security Certificate is Invalid or Does Not Match

Your users are frustrated that every 5 minutes, or upon opening Outlook, an obnoxious pop-up appears warning them that the Exchange server’s SSL does not match the FQDN. Danget…. This sounds like a poorly setup Exchange autodiscover URL! It can be incredible helpful to make a CNAME record such as “autodiscover.company.com” >> “remote.company.com”, phones suddenly become so much easier for users. We need to also set up a Microsoft Exchange UCC SSL Security Certificate.

Exchange-Autodiscover

Extremely common are the lazily set up domain names like “company.local”. Domain registrars no longer offer SSLs for .local, and Microsoft has been telling sysadmins for a decade to use full FQDNs for Active Directory, for example main.company.com, or city.joesbubblegum.net. This allows you to have resolvable SSL addresses for your various servers (web, mail, RemoteApp). This goes hand in hand with adjusting your autodiscover URL. It is very common to see internal Outlook clients resolve to “server.company.local” as their Exchange server. Once you make these changes, you may need to re-make their Outlook profile (fun stuff huh?) to refresh pulling the new FQDN for all future mail syncs.

Here is a cheatlist of commands to make the internal URL, and external URL, accepted by Exchange match.

You’ll need a UCC SSL, generally $150/year from GoDaddy. You can also used a self-signed CA if you don’t want to spend any money, but it can be a world of pain (always go Web Server with base64 on certsrv.asp if you do, that’s a good stickler that a lot of guides can mess up).

You’ll probably want to use Exchange Management Console to generate the certificate request. Those are separate tasks for separate articles. Generally the SAN (multiple FQDNs the SSL covers) would include “remote.company.com; autodiscover.company.com”. By including both under one umbrella, client devices can autoconfigure from anywhere, and no more SSL warnings.

First, find your existing settings in case you mess something up, or approval isn’t give for a UCC by management, or your need to go back for whatever reason.

## Changing Exchange 2010 to Use External DNS Name, Instead of .local
#Pull the old settings to be safe
Get-ClientAccessServer | FL
Get-WebServicesVirtualDirectory | FL
Get-OABVirtualDirectory | FL
Get-ActiveSyncVirtualDirectory | FL
Get-OWAVirtualDirectory | FL
Get-ECPVirtualDirectory | FL
Get-OutlookAnywhere | FL

Replace mail.yourdomain.com with whatever you normally use for mail resolution. You’ll also need to change the HostName of the Exchange server you’re using.

Watch out for the the switch “InternalHostname” at the bottom line if you do a search & replace command. Note if you don’t have Outlook Anywhere enabled it will just error out anyways (confirm with Get-OutlookAnywhere command).

Enter these commands into an Exchange Management Shell (Run as Admin!), then restart the transport service.

#Change hostname and resolved internal and external mail FQDN
Set-ClientAccessServer -Identity HostName -AutodiscoverServiceInternalUri https://mail.yourdomain.com/autodiscover/autodiscover.xml
Set-WebServicesVirtualDirectory -Identity "HostName\EWS (Default Web Site)" -InternalUrl https://mail.yourdomain.com/ews/exchange.asmx
Set-OABVirtualDirectory -Identity "HostName\oab (Default Web Site)" -InternalUrl https://mail.yourdomain.com/oab
Set-ActiveSyncVirtualDirectory -Identity "HostName\Microsoft-Server-ActiveSync (Default Web Site)" -InternalUrl https://mail.yourdomain.com/Microsoft-Server-ActiveSync
Set-OWAVirtualDirectory -Identity "HostName\owa (Default Web Site)" -InternalUrl https://mail.yourdomain.com/owa
Set-ECPVirtualDirectory -Identity "HostName\ecp (Default Web Site)" -InternalUrl https://mail.yourdomain.com/ecp
Set-OutlookAnywhere -Identity "HostName\Rpc (Default Web Site)" -InternalHostname mail.yourdomain.com -InternalClientsRequireSsl $true
#Restart Transport Service
stop-service MSExchangeTransport
start-service MSExchangeTransport

Migrating from ESXi to HyperV w/ MVMC

Migrating an ESXi Host to HyperV w/ Microsoft Virtual Machine Converter

I’ve been doing these a lot lately. Converting clients from ESXi free to HyperV on Server 2012 R2. This post is a bit unfair, as ESXi free is well, free, and Server 2012 R2 is $800. You could easily spend many, many thousands for ESX Standard/Essentials/Enterprise/etc.

 

This process uses the Microsoft Virtual Machine Converter freebie Powershell modules making for an ultra-fast conversion between VMDK to VHD.

 

Reasons we are converting hosts from ESXi free to Hyper V.

  • We take over IT from another MSP, who does a lazy and incompetent job that I get to clean up for my client.
    1. The ESXi hosts I usually see often have a poorly configured RAID. RAID5 for 8 disks for a SMB client too many times to count. Wiping it anyways affords this opportunity.
    2. Replacing PERC cards, (PERC310 – the worst RAID card I’ve ever used, not fit for a low-end-desktop).
    3. AD/FS/DHCP/DNS/Exchange/SQL all-in-one why not? Separation of roles for the client.
  • A 2012 R2 Standard license includes one HyperV host license, and TWO Server 2012 R2 Standard VM licenses. Since we are buying 2012 R2 for VMs to use under ESXi anyways, we already *paid for* a full-featured hypervisor, may as well use it.
  • More power, ESXi limits each VM to 8 vCPUs, while HyperV has no vCPU core limits. Older ESXi (<5.1) had a 32GB RAM limit.
  • Windows, I get a GUI with powershell, and don’t have to suffer through unclear and useless VMWare documentation and their awful esx-cli. If I need it minimal, just run Server Core.
  • Easy, easy, easy clustering and failover.
  • Inability to access the ESXi API for third-party apps with ESXi free. Generally an issue for backup software that isn’t provided by VMWare (Veeam), or making virtual-standbys (AppAssure).
  • Easier to make changes. We had to restrict ourselves to the old ESXi v8 hardware to easily make changes, any version higher and you have to use the web-editor or vCenter to make changes. Not fun to undo for a small business who can justifying buying vCenter.
  • Simpler remote-access. Able to use HyperV Manager, RDP in, or use an agent to directly access the HyperV host, no need for vSphere or vCenter. HyperV Host can have its own DNS to reach out to the internet if all the VMs need to be off.

 

The down and dirty, the process.

HAVE-A-BACKUP, Extract the VMDKs

We take a NAS on-site and make a share. As the vSphere storage download/file-browser utility can’t resume a download, can randomly crap out, and is slow, I install a FTP service via Putty and then use FileZilla to copy out ALL  VMDK files. Just switching from the vpxclient.exe VSphere app to FileZilla changes transfer rates from 35MBps to 90MBps. Confirm you don’t have any running VMWare snapshots.

#Enable SSH via vSphere Client
ESXi Host > Configuration > Security Profile > Services > Properties > SSH > Start

#Putty in to the ESXi host
cd /vmfs/volumes/DATASTORENAME/
mkdir ftp
cd ftp
wget http://esxi-customizer.googlecode.com/files/ProFTPD-1.3.3-8-offline_bundle.zip

#FTP in (FileZilla) your standard login, (usually root/password), and download the root folder containing all your files to the NAS (or USB 3.0 HDD), though we really only need the VMDKs.

$In the worst case, you can use the Datastore browser and try to download the VMDKs. Veeam's FastSCP is now built into their, no-joke, 1.2GB Veeam Backup installer that is pure bloatware.

 

 

When you are absolutely 100% sure you’ve got a full copy of ALL VMDKs, a backup, and your license keys written down.

Wipe the RAID

On a Dell Server, hit Ctrl+R on boot prompt to jump into the PERC RAID manager.

How you arrange your RAID virtual disks and spindles depends on the purpose of the server. SQL and RDS servers generally go RAID10. For a single simple DC and file-server, RAID6 is fine.

For this SMB client with only 600GB of data, with 5x 1TB HDDs and two VMS (SQL and DC/FS). We are going: RAID10 (4x) + 1x Global Hotspare.

RAID Virtual Hard Disks will be 100GB for HyperV host and its ISOs, then the remaining storage in another ~1.9TB disk for VMs.

 

Install Server 2012 R2, add HyperV Role, set a static IP/DNS, start Windows Updates.

 

Copy back your VMDKs to the HyperV Host.

Install Microsoft Virtual Machine Converter 3.0+. (https://www.microsoft.com/en-us/download/details.aspx?id=42497)

Here is the problem. MVMC is more meant for moving VMs with Microsoft System Center, and despite its name can make converting VMs very painful. However, the GUI is just a front-end for the installed-modules. Some simple code, and you will see a conversion run at the max speed your disks can handle 🙂

A big gotcha!

There are TWO .VMDK files that you need.

machinename.vmdk

machinename-flat.vmdk

The flat contains the data, the lesser file (usually 1KB) is the acting drive, that is a descriptor for the real data file. Run your command on the lesser file.

 

#If you've got anything below Win8/2012, use VHD. Win8/2012+ use VHDX.

#For converting standalone VMWare VMDKs into HyperV VHDs
#Powershell as Administrator
import-module ""C:\Program Files\Microsoft Virtual Machine Converter\MvmcCmdlet.psd1""
#Some Switch Options:
    -vhdformat vhdx (Needed for disks bigger than 2TB)
    -vhdtype FixedHardDisk (also called Thick Provisioning in VMWare)
    -vhdtype DynamicHardDisk (also called Thin Provisioning)
convertto-mvmcvirtualharddisk -sourceliteralpath "E:\HYPERV\VHD\machinename.local.vmdk" -destinationliteralpath "E:\HYPERV\VHD\" -vhdformat vhd -vhdtype DynamicHardDisk

Once it’s going, you’ll get a pretty powershell progress bar to slowly watch.

Migrating from ESXi to HyperV MVMC Powershell

Within HyperV Manager, create a new VM, pick the matching specs (vCPUs, RAM, NIC, etc), and add an already existing disk. The disk you just created!

Go ahead and boot the VM, expect a license-verification or re-activation as the virtual hardware just changed.

It is not uncommon for the *first* boot sequence to be very slow, especially for Exchange servers. You could just see a black screen in the HyperV Console for the VM. The VM is basically waiting for timeouts on some services before continuing, generally no longer than 15-20 nail-biting minutes. After boot, run the integration services disk to install your new virtual-hardware, then reboot the VMs. After you get through your the reboot and activation, your VMs should be happy and healthy.

Kicking Off the Site

Kicking Off the Site

If you’ve reached this website, you’re likely looking for a fix to a weird problem.

I’ve been a IT systems adminstrator for over six years, and want to offer simple and practical fixes to the problems I experience day-to-day. Google-fu has saved me many times before, hopefully my posts can help you out.

Also included are the exciting or tragic projects I come across. If any articles are helpful or interesting leave a comment, I’m always glad to hear stories.