Your Data and AI Firms

You may see news reports of our parent company, Automattic, striking deals to sell data from WordPress.com and Tumblr to OpenAI and Midjourney. The original report appeared in 404 Media and was picked up by The Verge. 

I want to assure WordPress VIP customers that your data has not been shared as part of any deal that Automattic may have negotiated and we will never share your data without explicit consent.

And while we’re on the topic of AI crawlers and your content – while it is not possible to prevent all AI crawlers, you can signal to reputable organizations that you do not want your content crawled. We have code examples for blocking AI crawlers in our documentation.

Please do reach out directly – I’m ng@automattic.com – if I can help in any way. Thank you for your continued partnership.

Incident Report: June 11 Service Interruption

On Tuesday, June 11, the VIP Go platform experienced a widespread service interruption that lasted for 3 hours and 11 minutes. On behalf of the team here at VIP, I would like to apologize for the impact this had on each of you. We place the reliability of our platform at the heart of everything we do and we take great pride in the trust you place in us each and every day. Please know that we take this incident seriously. I’d like to walk you through our understanding of this incident as well as the steps we’re taking to help prevent future occurrences.

What happened

A software update to the deployment system we use to apply updates to the underlying application software on the VIP Go platform introduced a defect that was at the root of yesterday’s incident. This resulted in client-specific WordPress application directories (e.g. plugins, themes, client MU plugins) being unavailable to the containers used to serve customer sites on VIP Go. Practically speaking, this means WordPress sites on our VIP Go platform displayed either a blank theme, a missing theme, or an incorrect theme.

The issue was resolved by reverting the update to the deployment system and then performing a rolling restart of all affected application containers across our platform.

Impact

• WordPress sites on the VIP Go Platform displayed either a blank screen, missing theme, or an incorrect theme.
• WordPress sites on the VIP Go Platform were operating without functionality provided by plugins, client MU plugins, or themes.
• Code deploys to Node.js sites on the VIP Go Platform would not have been fully executed, i.e. the code would be deployed but the deployed changes would not have come into effect.
• After the incident, some WordPress sites had to re-enable the correct custom theme, and also theme options (e.g. menu and widget assignments).

Timeline

• June 11th 2019 17:01 UTC: A change to our software stacks is deployed, which triggers the defect in the software stacks deployment process.
• 17:01 UTC: Alerting is triggered and investigation begins.
• 17:04 UTC: The change from 17:01 is reverted, but the issue is not resolved. Investigation continues.
• 17:14 UTC: A change in our deployment process is identified as the cause. Testing begins on the impact of reverting this change.
• 17:27 UTC: The deploy process change is reverted.
• 17:30 UTC: Software stacks are redeployed across our network.
• 18:00 UTC: Work begins to automate restarting all web containers safely across all data centers on a rack/switch basis.
• 18:21 UTC: Container rolling restarts begin and containers reconnect with their custom themes and plugins.
• 20:12 UTC: All sites on VIP Go are confirmed to be fully restored.

Future Prevention

We are working on implementing additional safeguards and process improvements designed to prevent similar issues from happening again. These include (but are not limited to):

• Updating Docker volume management to prevent disconnections.
• Deploying future changes in a limited scope to limit the effects of any uncaught issues.
• Updating our automated testing on releases prior to production rollouts.
• Updating internal tooling to facilitate faster global rollbacks.
• Disabling the default WordPress behavior of automatically switching themes when the current theme is not found (via the `validate_current_theme` filter).

Again, we sincerely apologize for yesterday’s interruption and appreciate the immense trust you place in us every single day. We are in the process of directly connecting with every customer who has been impacted. However, if you have any further questions or concerns related to this, please reach out to me (I’m ng@automattic.com), your VIP Relationship Manager, or open a ticket via vip-support@wordpress.com.

2016 VIP Workshop – Business Track Update

First things first – registration closes tomorrow! Have you registered yet? If not, do that now.

Second, we are very excited to update you all on what to expect as a part of the new business sessions at this year’s Workshop. We have focused on building a program that provides experienced perspectives on unique subjects that impact our products as well as our organizations. Here is a look at what you can expect in these new sessions:

• Exponential Results with Small Teams
• The Product Game
• Making the Most of Your Data
• The Convergence of Content, Design, and Technology
• Net Neutrality
• Emerging Market Internet

Read on for a bit more about each of these.

2016 VIP Workshop Update

We’re inspired by the response to the Workshop and are excited to have folks joining us from the biggest WordPress users across North America, South America, Europe, and Australia. Have you registered yet? Don’t delay as the registration deadline of April 7th is fast approaching and space is running out!

Register now

 

As mentioned earlier this month, we’re expanding the Workshop to include great material for developers and program leads. The final schedule details are coming together quickly and we’ll have more to announce on these next week. As a quick tease we’re excited to announce we’ll be kicking off this year’s the Workshop with the Emmy Award-Winning Xbox Live Director of Programming, Larry Hryb (a.k.a. Major Nelson).

Larry’s story of how a New England radio broadcaster transitioned into one of the most influential figures in the nearly $100B video game industry will set the tone for an incredible week in Napa. We are excited to see you there!

Larry Hryb, Microsoft Xbox’s Major Nelson joins us for this year’s VIP Workshop

Call for Entries – Best of Enterprise WordPress

 

We’ve recently opened registration for this year’s VIP Workshop. (Have you registered yet?)

Part of this year’s Workshop is a Best of Enterprise WordPress showcase where we’ll recognize the work of the amazing VIP community across a number of categories.

We’ve opened a call for entries for our Workshop attendees which will be open until April 15, 2016. We would love to recognize your best WordPress experiences and solutions from the past year, so please participate and we’ll see you in Napa!

Submit an entry

Announcing the 2016 VIP Workshop

 

Save the date (and don’t forget to register)! We are excited to announce the next WordPress.com VIP Workshop is coming May 9-12, 2016.

As the leading provider of enterprise WordPress solutions, WordPress.com VIP provides the premier event focused on the needs of enterprise WordPress users. New for 2016, the VIP Intensive Developer Workshop is now the VIP Workshop. We’ve updated the format to focus on both VIP product owners as well as developers, plus a new best of VIP showcase (stay tuned for when we open the request for entries).

Register now

 

The VIP Workshop provides a unique opportunity to learn from the WordPress.com VIP team in person, as well as exchange ideas and experiences with other WordPress.com VIP clients and partners through networking lunches and dinners, in-depth curriculum and exercises, and focused, collaborative conversations.

A quick peek at the itinerary – details & agenda will be continually updated on the WordPress.com VIP Workshop page.

• Monday, May 9th — Arrival Day. Check-in, special guest keynote and welcome dinner.
• Tuesday, May 10th — In depth business and engineering sessions.
• Wednesday, May 11th — In-depth product sessions and hands-on development workshops. We’ll wrap the day up with a best of WordPress.com VIP year in review!
• Thursday, May 12th — Departure Day.

Space is limited for this event, so register now and take advantage of early bird pricing! Early bird pricing is $3,500 each until March 15. After which, the full participant price will raise to $4,000.

Images by CM Howard from the 2015 Automattic WordPress developer summit at the Carneros Inn in Napa, CA

VIP at NAB Show in Las Vegas

Are you headed to NAB Show in Las Vegas this April? Members of the WordPress.com VIP team will be there along with a few of our colleagues from the Automattic family!

We’re organizing a dinner on the evening of April 14th (Tuesday), so if you or your colleagues will be there, please let me know by sending me a note – nick.gernert@automattic.com. I’ll be sure to get you details on the dinner as well as anything else we plan during the event.

We hope to see you there!

Headed to the CTO Summit in Miami?

Are you headed to the Innovation Enterprise CTO Summit in Miami this week? Members of the WordPress.com VIP team will be there and we’d love to say hi (we’re sponsoring the opening night reception).

If you or your colleagues will be there, please let us know by sending me a note at nick.gernert@automattic.com. I’ll keep you posted of any other events we organize and we’d love to catch up with you in person.

The Next VIP Developer Orientation + Town Hall is August 6th

This month’s VIP Developer Orientation + Town Hall is August 6th at 2pm 1pm ET!

The first part of the call is the VIP Developer Orientation, a quick 30-minute call for new developers and team members where a WordPress.com VIP developer will walk you through some of the tools, information, and other helpful tips which will help you do your job even better. We’ll cover frequently-asked questions about setting up your development environment, working with svn and submitting patches, how to get the most out of VIP Support and some of the VIP resources available to you.

Then at the :30 mark, we’ll switch gears slightly into the VIP Town Hall, which is open to all current clients & partners. You’ll hear what’s new with VIP in terms of features, functionality, and platform improvements, and we’ll be there to answer your questions as well. You can join for the entire call or for the part which interests you most.

Click here to register.

In Las Vegas for SXSWV2V?

If you happen to be in Las Vegas for this year’s SXSWV2V I’d love to say “Hi!”

Even better, you should come to the Monday Night Mixer that WordPress.com is throwing!

The mixer is in lovely Downtown Las Vegas at the Inspire Theater (501 Fremont St) from 6:30 til 8:30 tonight. I hope to see you there.