New Feature: Generate database backups on-demand via VIP Dashboard

We are delighted to unveil a noteworthy enhancement within the VIP Dashboard, aimed at significantly bolstering the flexibility of our database backup system. With this latest development, users now have the option to manually initiate backups for their respective databases. This addition, alongside our existing scheduled backup system, empowers you with even greater command over the safety and security of your valuable data.

Upon clicking the “Backup Now” button, a confirmation window will promptly appear, allowing users to proceed with the immediate initiation of a new backup.

It is essential to note that this feature is subject to a limitation in production environments, wherein only one backup can be executed within a 15-minute interval. Similarly, the backup frequency is restricted to once every 60 minutes for non-production environments.

This feature comes with a daily usage limit as well. In a production environment, users are allowed up to 12 requests within a 24-hour rolling window. For non-production environments, the limit is set to 4 requests within the same window.

Detailed information related to this feature can be found in our documentation.

If you have any questions about this change, please get in touch.

VIP CLI: Improvements to the Media Import tool

We’re pleased to announce a feature enhancement driven by customer feedback, which will allow you to move faster, by importing a larger media file archive in one go.

Going forward, the maximum size of media files archive that you can import with the vip import media command has been raised to 200 GB. This is an increase from the 60 GB limit that was in place prior to this.

You can read more about importing media into your environments over at our documentation.

If you have any questions, feel free to reach out, and we will be happy to help.

New Release: VIP-CLI 2.32.1

VIP-CLI 2.32.1 is now available.

v2.32.1 contains a hotfix for an issue that may prevent VIP Local Development instances from starting up after updating to Docker Desktop v 4.21.0 / Docker Compose v2.19.0.

v2.32.0 contains the following features/improvements:

Please run npm i -g @automattic/vip to get the latest improvements. You can find the detailed installation / update instructions on WordPress VIP Documentation site.

If you have any questions, please feel free to reach out, and we will be happy to help.

Completed: VaultPress Deprecation

We have reached the end of the VaultPress deprecation process. VaultPress is now disabled across the VIP Platform, and is no longer available for use on VIP environments.

You can continue to access your data through our convenient VIP Dashboard and VIP-CLI features, see below.

What does this mean?

If you’re a customer that has been using the VIP_VAULTPRESS_SKIP_LOAD PHP constant to continue using VaultPress to create backups for your environments, even after the deprecation on the 29th of March, you will no longer be able to do so. VaultPress will no longer be connected to your site environments, which means that no VaultPress backups will be generated. You will also lose access to the VaultPress Dashboard.

Customers that have not been using the VIP_VAULTPRESS_SKIP_LOAD PHP constant will also no longer be able to access the VaultPress Dashboard. VaultPress itself has been disconnected from your site environments since the 29th of March – so there have been no VaultPress backups generated since then.

Manage and export data with VIP tools

There are several ways to export and manage your data from the VIP Platform:

  • From the VIP Dashboard – choose to export the entire database, a single network site (for multisites), or specific tables
  • Using the  VIP-CLI command vip export sql

Managing data access through the VIP Dashboard and VIP-CLI allows us to provide a clear, simple, and secure service for your team. The VIP Dashboard user management system allows for regulation of access control, enhancing security and easing the workload for onboarding and offboarding processes.

All crucial interactions such as generation and download events are systematically tracked and recorded in the Audit Log, providing a clear record of significant system interactions. 

On-demand database backups

You can now trigger a database backup on-demand before exporting it using the VIP CLI.  The capability to trigger a database backup via the VIP Dashboard will follow soon.

Generating a backup for download will ensure that you are working with the most up to date data and should provide you all the tools needed to efficiently and confidently deploy applications. 

Please reach out to VIP Support with any questions or concerns you may have.

Feature Update: Export database backups generated on-demand

We’re thrilled to announce that we have expanded our VIP-CLI dev-env export sql command to include an option to generate a new backup and export it. This new feature is a step towards ensuring that you have access to the latest data of your application.

vip export sql @<your-application-id>.<environment-name> --generate-backup

The --generate-backup command option triggers the creation of a new database backup before the CLI downloads it onto your machine. If this option is not used, the database backup for your environment which was last generated by our systems will be downloaded (which is also the default behaviour of this command).

Here is an example of the new command option:

vip @0000.production export sql --generate-backup

NOTICE: A new database backup will only be generated for this environment if a recent backup does not already exist. Learn more about this: https://docs.wpvip.com/technical-references/vip-dashboard/backups/#0-limitations

Creating a new database backup...
✓ Preparing
✓ Generating backup
New database backup created
✓ Preparing
✓ Creating backup copy
✓ Requesting download link
✓ Downloading file - 100.00% (39.57 KB/39.57 KB)
File saved to /Users/home/2023-06-21T1404UTC-environment.go-vip.net-abcdefghijkl-db-backup.sql.gz

Please note that this feature has rate limits associated with it. Please take a look at the related documentation for more details on this.

To use this feature, please run npm install -g @automattic/vip to get the latest version of VIP-CLI. Verify that your VIP-CLI installation is on version 2.32.0 or above, by running vip --version from the terminal. You can learn more about this feature from our documentation.

We would love to know what you think. How do you hope to use these new tools? What could be added to improve your experience? Feel free to get in touch!

New Feature: Slow Query Logs

We’re pleased to announce the Slow Query Logs page that will help developers debug the database, providing the ability to identify queries made by an application that takes an unusually long time to execute.

Slow queries should be optimized to improve database efficiency and the overall responsiveness of an application.

Screenshot of new feature "Slow query logs"

A developer could get the information displayed on the new page, test the full query in the database, and work on SQL tuning to improve performance. Then, depending on the duration, rows examined, or rows sent, the developer can work on a different approach to improve it.

For example, a new index or limiting the rows returned can improve 99% of the query execution time and the page load.

Learn more about Slow Query Logs in our documentation.

We’d love to know what you think about this feature; please get in touch!

New Features: Simplified Database Export & Synchronization with Local Environment

We’re excited to release two new VIP-CLI commands that will help developers export and import a database backup into their local environment in a single step. The new commands will improve developer productivity and reduce errors coming from importing a database backup into a local development environment. This is a large improvement over the current multi-step process

New export command

We are adding the ability to export database contents of a site from VIP-CLI with a single command:

vip export sql @<your-site-id>.<environment-name> –output=<output-file>

This command automatically detects the latest backup of a site, prepares SQL export, and downloads the file as specified with the –output option. If this option is not specified, it downloads in the user’s current working directory.

$ vip export sql @0000.production --output /home/user/exported.sql.gz
Fetching the latest backup for mytestsite
✓ Latest backup found with timestamp 2023-05-30T08:16:09.000Z
Creating a new export for the backup with timestamp 2023-05-30T08:16:09.000Z
✓ Preparing 
✓ Creating backup copy 
✓ Requesting download link 
✓ Downloading file
File saved to /home/user/exported.sql.gz

Please note that this command does not let you select any specific backup, tables or network sites. We will iteratively add features to it, and for now it can only export the latest backup, all tables and all network sites.

New sync command

To try this feature, ensure that you’re on the latest version of VIP-CLI and simply run the following in your terminal:

vip dev-env sync sql --slug=<your-local-env-slug> @<your-site-id>.<environment-name>

Depending on the size of the database backup, this command may take a bit of time to finish. A successful attempt should look like the following:

$ vip dev-env sync sql --slug=0000-production @0000.production
Fetching the latest backup for mytestsite
✓ Latest backup found with timestamp 2023-05-30T12:43:07.000Z
Creating a new export for the backup with timestamp 2023-05-30T12:43:07.000Z
✓ Preparing 
✓ Creating backup copy 
✓ Requesting download link 
✓ Downloading file - 100.00% (37.01 KB/37.01 KB)
File saved to /tmp/vip-cli-YrjINx/sql-export.sql.gz
Extracting the exported file /tmp/vip-cli-YrjINx/sql-export.sql.gz...
✓ Extracted to /tmp/vip-cli-YrjINx/sql-export.sql
Extracting site urls from the SQL file...
Running the following search-replace operations on the SQL file:
  //mytestsite.go-vip.net -> //0000-production.vipdev.lndo.site
✓ Search-replace operation is complete
Importing the SQL file...
Success: The cache was flushed.
Success: Created user 3.
Success: User "vipgo" created.
✓ SQL file imported

Please note that this does not yet have full parity with the feature on the VIP Dashboard. We currently support only importing the latest backup of the entire database, and only single site installations.

To use this feature, please run npm install -g @automattic/vip to get the latest version of VIP-CLI. Please also verify that your VIP-CLI installation is on version 2.30.0 or above, by running vip --version from the terminal.

We would love to know what you think. How do you hope to use these new tools? What could be added to improve your experience? Feel free to get in touch!

Changes to Log Shipping and Database Backup Shipping Configuration

Today we are announcing two modifications which are designed to enhance the security of the HTTP Request Log Shipping and Database Backup Shipping configuration processes.

Instead of relying on the direct application of an AWS S3 bucket access policy, we now provide an AWS CloudFormation template that grants our tools the necessary access to your AWS S3 buckets for shipping HTTP logs and database backups. This improves the management of the access policies required to allow our systems to communicate securely with your S3 buckets.

Additionally, we’ve added a new input field called AWS Account ID. This is embedded in the CloudFormation template and helps validate that the specified bucket is owned by your AWS account, to avoid any misconfiguration issues.

You can follow this AWS guide to apply the AWS Cloud Formation template in your AWS Console.

If you have already enabled either HTTP Request Log Shipping or Database Backup Shipping, no changes are necessary and they will continue to work as configured.

If you would like to reconfigure any active HTTP Request Log Shipping or Database Backup Shipping configurations, you are welcome to do so by simply disabling and then re-enabling each shipping configuration. Once re-enabled, you may remove the Access Policy that you have previously applied to your bucket.

You can find more information on configuring both these features in our documentation:

If you have any questions about this change, please get in touch.

New in Beta:  Block Data API

We’re pleased to announce the beta availability of a new Block Data API.  It is now released for you to begin testing.  

What is the Block Data API?

Animation detailing how the block API functions

For many use cases, it’s helpful to access blocks in WordPress as pure JSON data instead of HTML.  Doing so eliminates the need to write parsing layers to transform markup into JSON, and to keep those translation layers up to date with every version of Gutenberg.  All of this saves developer time and effort.  

While options have existed for accomplishing this with GraphQL when accessed via REST, translation layers were required. The Block Data API is a new REST API endpoint to retrieve the Gutenberg blocks of a single WordPress post.

What are some use cases for the Block Data API?

There are several known use cases for the Block Data API.  These include:

  • Power your Node.js front end with less reliance on dangerouslySetInnerHTML. 
  • Send data about your blocks to Machine Learning and AI applications.
  • Use blocks in mobile applications with reduced processing.

This is not a comprehensive list.  As we go through the beta period, we’d love your feedback on how you use the Block Data API, what works well for you, and what you’d like to see improved.

While the API is in beta, you can offer that feedback by opening a GitHub issue. When it is released, we can provide support via normal means.

How do I get access to the Block Data API?

To install the plugin, go here and follow the instructions that have been provided

Where can I find more information/documentation?

The project Readme has comprehensive information on the plugin and its functions.

Upcoming: VaultPress Deprecation

Over the next few months, we will be deprecating and then removing VaultPress from the WordPress VIP platform. The VIP Dashboard now supports exporting database backups for all your applications and environments, the primary feature previously provided by VaultPress.

Here’s the timeline for the planned deprecation and removal:

  • Tuesday, March 14th, 2023: VaultPress will be disabled on non-production environments
  • Tuesday, March 28th, 2023: VaultPress will be disabled on production environments
  • Tuesday, July 04th, 2023: VaultPress plugin code will be removed from the VIP Platform’s Must Use (“MU”) plugins and the VIP Platform

Once VaultPress is disabled for an environment, you won’t be able to access database backups from the VaultPress Dashboard. Instead, the VIP Dashboard will provide you an integrated experience for managing exports that fits in with your existing VIP workflows. 

If you’d like to continue leveraging VaultPress until the final removal date in June, you can add the following line to your vip-config.php file:

define( 'VIP_VAULTPRESS_SKIP_LOAD', false );

With this in place, VaultPress will continue working on your environments even after we deprecate it on non-production and production environments. It will continue working until its removal date at the end of June.

Managing exports within the VIP Dashboard

We’re excited to have these and more new features coming soon to WordPress VIP.

  • Database exports are optimized for use in development environments and allow for further customization and filtering.
  • Security is enhanced by including user access control for backups under the VIP Dashboard user management system, easing onboarding and offboarding workload.
  • Tracking generation and download events are available in the Audit Log.
  • Backup exports via VIP-CLI (upcoming feature)
  • Easier syncing of data into the VIP Local Development Environment (upcoming feature)
  • On-demand / ad-hoc backups (upcoming feature)

Please reach out to VIP Support with any questions or concerns you may have. We are happy to accommodate an extension should you need it.