How to: Invalidate the VPLEX Cache

This is a brief note about a new capability we have introduced starting GeoSynchrony 5.2. I have been seeing a bunch of questions on our internal mailing lists about this. Hopefully, this note addresses a bunch of them.

Why do you need it?

Quite a few VPLEX Local and Metro customers use local replication on backend arrays – as an example, if the backend array is a VMAX, this would be using TimeFinder Snaps and Clones. The same is true for all supported EMC and non-EMC arrays on VPLEX. In the lifecycle management of these local replicas, you often need to expose a copy of the data as the original volume itself (when recovering from operational errors or going back to a different point in time). From a host standpoint, this volume would be indistinguishable from the original volume – it has the same volume identifier etc.
Let me now draw up the problem statement for you.
VPLEX Local and Metro have a read cache. In other words, for a given volume / consistency group, VPLEX will store data so as to enable cache access for that volume / consistency group without having to access the backend. This cache access is accurate assuming that the media underneath (i.e. the storage volume from the backend array) is not changed.
For such a volume exposed to the host, when you decide to mount the replica for that volume and present it to the host, in effect, the underlying media has been altered. This renders (unavoidably) that the VPLEX cache is out of sync with that underlying volume (since the cache may be written later than the replica timestamp). Any host that is likely to access the volume is likely to detect an inconsistency in the data. If you like analogies, this is equivalent to the disk drives being changed on your computer while the applications are running. Yeah, BAD idea.
Up until GeoSynchrony 5.2, the way to address this was to remove the virtual volumes for which the replica was mounted from the storage view and then re-add that volume back in. This would force VPLEX to delete any cached data associated with that volumes – allowing the volume to be consistent with the backend storage. And, in case you are wondering, yeah – not our favorite procedures either :-).
So, starting GeoSynchony 5.2 (released in May this year), we introduced the cache invalidate command which performs the invalidation function described above without having to remove and re-add the volume into the storage view. Let’s look at the command in more detail.

How the command works

The cache-invalidate command operates in the virtual volume as well as the consistency group context. In the virtual volume context, the command clears the cache for that virtual volume on _all_ directors where the volume is made visible within that cluster. In the consistency group context, the same operation as above is accomplished except for all volumes in the consistency group across all VPLEX directors within the cluster where the consistency group (or volumes from it) are made visible. Once you do this, any subsequent read for the volume / consistency group will result in a cache miss and will be performed off the backend array.
So, what does the command look like
  • virtual-volume cache-invalidate [-v |–volume] virtual-volume –force
  • consistency-group cache-invalidate [-g |–consistency-group ] consistency-group –force
  • Things to remember

    There are a few things to remember while executing this command:

    • This command will suspend host I/Os while the cache is being invalidated. Ideally, host I/Os should be stopped prior to invalidating cache. Remember that suspending I/Os on a live system could cause the host to detect a data unavailability (depending on how timing sensitive it is)
    • This command performs pre-checks to see if the volume is healthy on all directors before executing. If the volume is not healthy, you will get warnings about this.
    • The VPLEX cluster should not be undergoing an NDU while this command is being executed
    • There should be no mirror rebuilds or migrations in progress for virtual volumes affected by this command
    • No volume expansions (more about this in a later post) when this command is executed

    Checking the status of the command

    We have introduced a supplemental *get* style command to help customers understand what the current status of the cache-invalidate command when executed.
  • cache-invalidate-status [-h | –help] [–verbose] [-v | –virtual-volume=]
  • The output from the command will have the following potential values:
    • Status
      • in-progress: cache-invalidate is running.
      • completed: cache-invalidate has completed execution successfully.
      • error: cache-invalidate has finished but with error.
      • unknown: This status is reported when a particular director where the command needed to be executed cannot be reached
    • Result
      • successful: cache-invalidate completed successfully.
      • failure: cache-invalidate finished with failure. Cause field indicates the appropriate reason
      • error: cache-invalidate is in error state.
    • Cause
      • This is a text string indicating the reason for any failure / error during execution.

    And from there you are good to go! Happy cache-invalidating! If you have any questions, please post them in the comments section.

      Update:

    Don Kirouac (@kirostor) made me aware via twitter of a whitepaper that describes the usage of cache invalidate. Thanks Don!

    Finally, to all readers of this blog, a very Happy Holiday Season to you and your near and dear ones and Best wishes for 2014! We have some exciting stuff coming your way courtesy of the engineers working on VPLEX and RecoverPoint. And I cannot wait to tell you all about it … Stay tuned!

    Advertisements

    5 thoughts on “How to: Invalidate the VPLEX Cache”

    1. I understand that the VPLEX cache gets used almost exclusively for host read requests. Even if the target host that has say, a Timefinder Cloned volume mounted on it, it is only writing to the volume (the VPLEX virtual volume). The application reads are satisfied by the VPLEX cache. Why does the host-cache come into picture? I mean, why does the host need to do intelligent buffering from it’s end in the first place?
      re-phrasing. The write to the underlying VPLEX virtual volume outside of the VPLEX IO path should have no impact on the VPLEX’s ability to cache data that will be used by an application for a read, right?

      thanks.
      rg.

      1. Hi rg

        Thanks for reading the post. Let me make sure I understand the question you are asking.

        The configuration you are describing is where the primary backend volume is exposed to the host through VPLEX and where the TF Clone is mounted directly to the host without VPLEX being in between.

        Assuming this is the configuration, let me try and answer your questions.

        • 1. VPLEX cache is largely complementary to the host cache. The host cache is used to reduce the need for the host to go to storage on the SAN in the first place.
        • 2. For the volume that is exposed directly to the host, you are correct. The write to the mounted clone should have no impact on the ability of VPLEX to cache.

        If I have misunderstood the configuration or if there are follow up questions, please let me know.

        Regards
        Ashish

    2. If the underlying storage device is left untouched by VPLEX when using back-end replication technologies like Timefinder clone, are the host reads for the Timefinder Cloned target volume still satisfied using the VPLEX read cache?

      rg.

      1. If the volume is presented to the host without VPLEX in the data path, then host reads are not satisfied using VPLEX.

        One important thing to remember, it is _NOT_ okay to have a given volume exposed to the host through VPLEX and on another path, directly to the host without a path through VPLEX.

        May be a drawing of the configuration you are thinking of will help resolve this if there is any confusion. I have also posted a link to a white paper that describes what configurations can be used to help clarify.

    3. hi,
      This question is not related to cache-invalidation, but related to EMC VPLEX. a distributed device on VPLEX metro, has both source and target as RW enabled, so if the hosts on both ends try to write the same block on source and target at same time, how does the write happen? which write is given precedence over the other?

    Leave a Reply

    Fill in your details below or click an icon to log in:

    WordPress.com Logo

    You are commenting using your WordPress.com account. Log Out / Change )

    Twitter picture

    You are commenting using your Twitter account. Log Out / Change )

    Facebook photo

    You are commenting using your Facebook account. Log Out / Change )

    Google+ photo

    You are commenting using your Google+ account. Log Out / Change )

    Connecting to %s