Introduction to NetWorker with CloudBoost
NetWorker with CloudBoost enables long term storage provisioning via the cloud.
NetWorker sends a backup clone to the CloudBoost virtual appliance. The CloudBoost virtual appliance translates these into generic objects which are sent to an object store, which can be public, private, or hybrid. The CloudBoost virtual appliance presents itself as a NetWorker Advanced File Type Device. The enabled workflow is a clone operation to the cloud; it is not a backup to the cloud. With this low cost tape replacement solution, each CloudBoost virtual appliance can support up to 400 TB of addressable back end storage.
The CloudBoost virtual appliance decouples metadata from data, which removes a bottleneck for cloud reads and writes. Encryption keys, metadata, and file system information are housed separately from the data. All advanced data services, such as chunking, encryption, in-line de-duplication, compression, and bulk data transfers are performed separately from the metadata.
Individual CloudBoost deployments can support only one target object store. When a cloud object start is selected and the share data store is configured, the CloudBoost virtual appliance is locked to that target. To change object storage targets, the CloudBoost virtual appliance must be re-deployed.
System architecture and components
The CloudBoost virtual appliance is deployed as a single virtual machine (VM) comprised of these core components.
- The 8.2.1 versions of the NetWorker Storage Node and Advanced File type Device are pre-installed on the Cloudboost VM as the primary transport mechanism for cloning backups from local NetWorker targets to Cloudboost.
- The management server is a monitoring component for collecting instantaneous and historical statistics. It also provides a web-based console for system management.
- The discovery service allows the CloudBoost clients to locate and access the shares and learn the IP address of the metadata Server exporting them.
CloudBoost sizing and performance considerations
You can find information about CloudBoost sizing, performance and requirements here.
The CloudBoost virtual appliance requires 750 GB of internal capacity for storing CloudBoost metadata. SSDs are recommended for optimal performance. Additionally, the virtual appliance requires a single 4-core CPU with minimum 16 GB of memory (32 GB of memory recommended).
De-duplication and cloud capacity
A CloudBoost virtual appliance supports up to 400 TB of addressable back-end capacity in object storage. This is the total amount of unique capacity in the object store after deduplication. Based on preliminary test data, CloudBoost expects to achieve a 2x—4x range of de-duplication. Backups of file systems, applications, and databases where file sizes are typically small are expected to achieve close to 2x de-duplication on average. Backups of virtual machines where typical virtual disks sizes are larger could see up to 4x de-duplication. Based on this range of de-duplication, each CloudBoost virtual appliance Introduction to NetWorker with CloudBoost can support 800 TB—1.6 PB of logical client capacity. That said, proof of concept testing, or testing with up-to-date, real data is recommended.
WAN bandwidth is expected to be the most common bottleneck. A properly-resourced CloudBoost virtual appliance can saturate a 1Gbps link with 30ms RTT latency without hitting any limits within the VM itself. Object store ingest limits are another potential bottleneck. In some cases we reach the objects/sec limit that can be sustained by a single logical container in the object store.
Minimum WAN requirements
EMC recommend a minimum bandwidth of at least 10Mbps to the cloud with a maximum latency of less than 100ms RTT for the CloudBoost solution. Extremely low bandwidth links may result in backup and restore timeouts.
Multiple clone sessions
For parallelism, EMC recommend creating multiple AFTD devices under the /mnt/magfs/ base mount point within CloudBoost, and using one clone session per device. One session per device is recommended for optimal de-duplication. Multiple clone sessions to the same device can result in lower de-duplication ratios and longer clone times.