An early Christmas gift from Veeam. If you have a title / certification mentioned in the title of this note, you qualify for a free 2-socket Veeam Backup & Replication license. Apply here. MCTs, MCTSs and MVPs can get their copy here. In both cases however you can choose key type as Both (VMware and Hyper-V) so I guess the license for VMware and Microsoft certified is the same.
...now browsing by category
I’ve had recently a talk with Veeam representative and I’d like to share some interesting resources I have received (this time all in Polish
and some movies:
Failed to validate command "dd if="/vmfs/volumes/<path to vmdk>" count=1 bs=16384" feedback
/bin/dd if="/vmfs/volumes/<path to vmdk>" count=1 bs=16384 returned non-zero code
- dd if="/vmfs/volumes/<path to vmdk>" count=1 bs=16384/bin/dd: opening `/vmfs/volumes/<path to vmdk>': No such file or directory
Ok, so my <path to vmdk>, lest’s call it DatastoreA was not correct in that it did not contain the vmdk. The server was up and running with access to all disks. So where did Veeam take these settings from?
I opened VM’s properties and checked where the vdisks were configured to be. Yeap, one disk is supposed to be on DatastoreA but it is not, there are only logs, vmx file, etc.
A quick glance on vmx and the VM reveals its secret. The problem was caused by the following line:
fileSearchPath = "/vmfs/volumes/DatastoreA/MyServer;."
So that’s it! Honestly I don’t know why this entry appeared in VM’s vmx file. I did not mean to use an absolute path to point to the disk. Veeam does not seem to use it either.
In the end I modified the vmx manually, run the VM and the backup completed successfully.
I encountered the problem in Veeam 3.0 (yes, old one, I know). Do you know if in newer versions fileSearchPath parameter is used by Veeam?
Here‘s a discussion about fileSearchPath (although in VMware Workstation).
What’s new under the hood?
- Free e-discovery and item recovery for Microsoft Exchange
- Easy VM recovery from SAN snapshots
- Advanced monitoring, reporting and capacity planning (requires Veeam Management Suite 6.5)
- New hypervisor support: VMware vSphere 5.1 and Windows Server 2012 Hyper-V
- and more!
When you use an account to set up a SOAP connection to vCenter Server:
you need to assign to these account certain privileges. The easiest way to do so is to create a role in vCenter and assign it to this account. The role will need to get the following permissions, according to backup type you intend to use. This table has been taken from this great Veeam blog note. Check out this article should you want to know how to set up VSS-enabled backups and what else you need for successful Veeam backups.
|Privilege Level||vStorage API Virtual Appliance mode||vStorage API Network mode||vStorage API SAN mode|
|Global||Log event||Log event||Log event|
|Datastore||Low-level file operations||Low-level file operations||Low-level file operations|
|Virtual Machine state||Create SnapshotRemove Snapshot||Create SnapshotRemove Snapshot||Create SnapshotRemove Snapshot|
|Virtual Machine configuration||Disk change trackingChange resourceAdd existing diskRemove disk||Disk change tracking||Disk change trackingDisk lease|
|Virtual Machine provisioning||Allow read-only disk access||Allow read-only disk access||Allow read-only disk access|
I like problems. While solving them we can always learn something new and extend our knowledge. Today’s problem symptoms in Veeam’s failed job are as follows:
Can't process object "VM name".
Object "VM name" not found.
This means that VMs have been probably removed from inventory or the host has been removed from vCenter and then readded. This operation changes the ID of the VM and Veeam stops recognizing it. We are not talking about BIOS UUID written in vmx but rather about Managed Object Reference ID, the identificator inside vCenter „reality”, generated for new objects by vSphere when objects, like VMs, are created. Before fixing Veeam let’s have a look on different identifiers in vSphere / on ESXi.
These are three different IDs we can have:
1 .BIOS UUID – written in vmx file, it is not guaranteed to be unique.
2. instanceUuid – introduced in the vSphere 4.0 API to resolve problems with ununique BIOS UUID.
This a part of vmx file:
In this file we can see two identifiers – uuid.bios (BIOS UUDI) and vc.uuid (instanceUUID). I could not find any information in VMware documentation that vc.uuid is the same as instanceUUID but it can be confirmed with this Pearl script:
3. Managed Object Reference ID (MoRef ID) – as mentioned before this is an ID for use in vSphere.
We can check it with the same Pearl script as before or simply withPowerCLI:
Now I will remove the machine from the inventory:
When I readd it and check the ID, it is different:
This is the ID that is used by Veeam Backup & Replication 4. It does not happen often to remove a VM from inventory or remove host with this VM running from vCenter. But when it happens it will break the backup job.
Here is a solution to the problem in Veeam 6 that applies manual change to Veeam database.
I found a fix suggested by this KB much easier. It was enough to edit properties for the job, remove VM and readd it. The backup file remains the same (it is greyed out and it cannot be modified anyway). My only concern was if the backup is not started from the scratch. I run a modified job on a test server and sure enough the restore was possible from the backups before the last one.
After you had upgraded ESX 3.5 hosts to 3.5 update 5a, you may find that Veeam backup stopped working. The error message is:
Checking the license for the source host ‘xxx.xxx.xx.x’
The request refers to an object that no longer exists or has never existed.
A quick search and I found the following Veeam KB: http://www.veeam.com/KB1063
Seems that the upgrade modifies hosts’ UUID and Veeam Backup stops recognizing them as licensed servers. The KB presents three solutions:
- Recreate jobs
- Click On Help|License Information|Licensed Hosts, and revoke the license for the problematic host (or all hosts).
- Using SQL Studio Manager, run the following script against the ‘VeeamBackup’ database: delete from [dbo].[HostsByJobs]
The first one in my case was not useful – in Veeam Backup and Replication it seems not to be possible to map new jobs to old backup (in order to continue old backups) or at least I could not find one.
Solutions 2 and 3 are basically the same thing – revoke current licenses and let servers with new UUID register again. Since they do the same thing, we won’t get dirty with messing around directly with the DB.
Go to Help > License Information and click on Licensed hosts:
Select your hosts and click Revoke. To add the host back as licensed you don’t have to do anything. Just run your existing jobs and ESX(i) will be registered as long as you have enough slots within your license. Also, the jobs will succeed.
Tip: One time I had to close the console and restart Veeam service after I revoked the license and before the servers were added back. Another time it went smoothly without restarting the service.