4.2. Reconciliator’s Algorithm¶

Warning

Still in progress

4.2.1. Recurrent purge¶

From time to time, the recurrent purge is there to clean up the database:

Purge a “Deleted” object (really deleted) once the “expiry” date is over
Once the object reached its expiry date:
- Possibly moving it to a dedicated Bucket for archiving with possible new Expiry date
  - If archival is enabled, only one bucket is dedicated to archival process per client
  - Archival is only valid for object with a valid Driver Object
- Or run “Delete” operation, setting a new “Expiry” date for later on purge
- In all case, send a message to replicate the deletion order to other sites

Those actions should not be done during reconciliation process but at least before.

Recurrent Purge on Expired date¶
From	Ready	Deleted
		Once expired
Object DB		x
Object Fix		Purge (check Driver)

	Once expired
Object DB	x
Object Fix	Archive (Move to another Bucket if enabled and exists) or Delete

4.2.2. Reconciliation¶

Local Reconciliation is in several steps:

Clean step on Objects and Native local objects status according to Request filter
Start a new Native local objects for Request, from previous run if any
Snapshot: Fill or Update Native local objects according to Request filter using Objects database
Snapshot: Fill or Update Native local objects according to Request filter using Driver contents (probable longest duration)
Create Site Listing from Native local objects and update if necessary Objects accordingly
Get Site Listing for remote Reconciliation site, the one that starts this reconciliation process

Optional steps for Local Reconciliation:

Clean Native local objects if not to be kept for later use
Once validate remotely, Clean local Site Listing if not to be kept for later use

On site where this process is started:

Create the Request context
Send order of local reconciliation process on each remote site and itself
For each remote site, get the remote Site Listing to append to the local therefore global Site Listing
Final Reconciliation step

Final Reconciliation steps:

For all entries in Global Site Listing:
- Compute Site Action for All sites / Partial sites context
For all Site Actions, transfer replication order accordingly to each site, even itself
- Sites having READY ACTION will be considered as remote source site for other sites
Once all transferred, clean Site Actions
Statistics update using previous steps

Then reconciliation corrections happen using standard replication order, using special status to act according to the needs:

DELETE ACTION: will delete if possible MD and Driver or both
UPDATE ACTION: will update Object only from remote site, Driver being ready
UPLOAD ACTION: will update Driver (content) and possibly Object (some metadata) from remote site

4.2.2.1. Clean step¶

Clean step is several sub-steps: on Objects and Native local objects status according to Request filter

All Unknown status Objects and Native local objects are removed
Update Objects reconciliation status from UPLOAD/ERR_UPL to TO_UPDATE
Update Objects reconciliation status from DELETING/ERR_DEL to DELETING
Update Objects reconciliation status from READY or DELETED to respectively the same
Purge Native local objects with status UNKNOWN, UPLOAD, ERR_UPL, DELETED or ERR_DEL

Pre Reconciliation Purge¶
From	Unknown	Upload	Ready	Err Upl	Deleting	Deleted	Err Del
Purge
Object DB	x
Object Fix	Delete

Purge	status	older	than	reference
Object DB		x		x	x		x
Object R Status		To Update	X	To Update	Deleting	X	Deleting

Pre Result Reconciliation Purge¶
From	Unknown	Upload	Ready	Err Upl	Deleting	Deleted	Err Del	To Update
Previous Result		x	x	x	x	x	x	x
Fixed Result	Removed	Removed	x	Removed	x	Removed	Removed	x

4.2.2.2. Snapshot step¶

Two steps are concerned:

Fill or Update Native local objects according to Request filter using Objects database
- If the reconciliation status is a “delete” status, the driver part is ignored
Fill or Update Native local objects according to Request filter using Driver contents (probable longest duration)
- Will add or update the Driver part

Load from DB and Driver¶
From	Unknown	Upload	Ready	Err Upl	Deleting	Deleted	Err Del	To Update
From DB			x	x		x	x	x
From Driver			x

4.2.2.3. Local Reconciliation step¶

Create Site Listing from Native local objects and update if necessary Objects accordingly:

From Driver only, consider Object shall be READY and To Update
- Create missing Object with existing metadata from Driver (possibly some missing)
From Db only, consider Delete like as Deleted, and others (Object shall exist) as To Upload again
- Update Objects accordingly
From both, consider Delete like as To Delete, and others (Object present but not ready except READY ones) as To Update (metadata only)
- Update Objects accordingly

Fix LocalSite Reconciliation: Driver present, DB absent¶
From	Ready	To Update
Source Sites	x
Object Fix	X (new one)	+X
Updated Sites	Update

Once done, the to update ones will be update from the Driver and set as Ready.

Fix LocalSite Reconciliation: DB present, Driver absent with Available like status¶
From	Upload	Ready	Err Upl	To Update
Sites	x	x	x	x
Object Fix	X			+X
Fix Sites	Upload

Fix LocalSite Reconciliation: DB present, Driver absent with Delete like status¶
From	Deleting	Deleted	Err Del
Sites	x	x	x
Object Fix		X
Fix Sites	Delete	Deleted	Delete

Fix LocalSite Reconciliation: DB and Driver presents with Ready like status¶
From	Upload	Ready	Err Upl	To Update
Object DB	x		x	x
Object Driver		x
Object Fix		X		X
Sites Fix	Update		Update	Update

Object DB		x
Object Driver		x
Sites Fix		Ready

Once done, the to update ones will be update from the Driver and set as Ready.

Fix LocalSite Site Reconciliation: DB and Driver with Delete like status¶
From	Ready	Deleting	Deleted	Err Del
Object DB		x	x	x
Object Driver	x
Object Fix		X
Sites Fix		Delete	Delete	Delete

4.2.2.4. Final Reconciliation step¶

From all remote Reconciliation site listing, Actions are sorted according to descending event dates, the latest being the primary event.

Thr order of actions is: DELETE > READY > UPDATE > UPLOAD

So for instance:

latest event: DELETE like and anything else
- => DELETE everywhere
latest event: READY like (UPDATE/UPLOAD)
- => UPDATE or UPLOAD from READY site(s) (potentially multiples sources)
- Special case: if none are READY, UPDATE ones will changed to READY
- Special case latest event: all UPLOAD status (no READY or UPDATE)
  - These final cases are in big trouble since there is no more available correct information
    - UPLOAD cannot be fixed if there is no source at all => changed to ERROR_ACTION with no source to get ERR_UPL status

Two cases have to be checked: all sites or subset of sites are referenced for each item:

One entry has all sites referenced: so all know about it
One entry has a subset of all sites referenced: therefore, except for delete action where they are ignored, they should be considered as an UPLOAD action (for UPDATE, the concerned site will upgrade locally to UPLOAD since no object present)

Those 2 cases are fusion in one:

For all Site Actions, transfer replication order accordingly to each site, even itself
- Sites having READY/UPDATE ACTION will be considered as remote source site for other sites

Compute Remote Site Action Reconciliation¶
From	DELETED	DELETE	READY	UPDATE	UPLOAD
Primary	X	X
Other	°	°	°	°	°
Actions	DEL		DEL	DEL	DEL

Primary			X
Other	°	°	1	°	°
Actions	UPL from (1)X	UPL from (1)X		UPD from (1)X	UPL from (1)X

Primary				X
Other	°	°	1	2	°
Actions	UPL from (1) xor from (2X)	UPL from (1) xor from (2X)		UPD from (1) xor UPG(2)	UPL from (1) xor from (2X)

Primary					X
Other	°	°	1	2	°
Actions	UPL from (1) xor from (2)	UPL from (1) xor from (2)		UPD from (1) xor UPG(2)	UPL from (1) xor from (2)

Primary					X
Other	°	°			3
Actions	invalid	invalid	invalid	invalid	ERROR

Warning

Transfer replication order and application not yet implemented

Remote Site Action final Reconciliation¶
Object DB	UPLOAD	READY	ERR_UPL	DELETING	DELETED	ERR_DEL	ABSENT
Actions	DELETE
Object DB Fix	DELETED	DELETED	DELETED	DELETED	invalid	DELETED	ignore
Driver Fix	DELETE if exists

Actions	UPDATE
Object DB Fix	READY If Driver	READY If Driver	READY If Driver	invalid	invalid	invalid	READY with upload
Driver Fix	Upload if needed	invalid	Upload if needed				Upload

Actions	UPLOAD
Object DB Fix	READY	invalid	READY	READY	READY	READY	READY
Driver Fix	Upload	invalid	Upload	Upload	Upload	Upload	Upload

Actions	UPGRADE
Object DB Fix	READY	invalid	READY	READY	READY	READY	READY
Driver Fix	Upload if needed

Actions	ERR_UPL
Object DB Fix	ERR_UPL	invalid	ERR_UPL	invalid	invalid	invalid	invalid
Driver Fix	Nothing

4.2.3. Special Reconciliation modes¶

Two special cases are implemented:

Initialization from existing object in Driver Storage while CCS was not yet used to create them
Initialization for a new site (whatever really new one or disaster one so almost new), in order to speed up reconciliation step for this new site from an existing site

4.2.3.1. Initialization from existing Object Storage without CCS¶

When moving an existing application with existing Objects to Cloud Clone Store, one could use the following batch:

From Storage Driver, initialize Objects and Buckets in database according to arguments
- Arguments such as: bucket name, client Id to use, common specific metadata

Note that the issue right now identified is that Bucket are named using clientId within CCS. To enable such an import, a special attention should be done on this case (where bucket does not have ClientId in its final name).

All items will have READY status.

4.2.3.2. PRA reinitialization or new site initialization¶

When a site has a disaster (partial or full disaster) or when a new site is added to an existing multi-sites CCS configuration, there is a special batch to resume the CCS database and Cloud Storage contents.

Once the CCS is installed (or reinstalled), instead of running a standard Reconciliation, one can run this specific Reconciliation from existing (or none) status on the new/rebuild site.

Mode empty site: no objets neither storage objects in the site to synchronize
- This mode is optimize for “all” synchro mode with no control on destination site since nothing is there
- ALl items will have READY Status using UPLOAD_ACTION from given existing sites
Mode disaster recovery: objects or storage objects can exist, partially
- This mode is optimize for “all” synchro mode with control on destination site since objects or storage objects or both can exist
- ALl items will have READY Status using UPGRADE_ACTION from given existing sites

Navigation

Related Topics