InfoSphere CDC - Product Development Voice!


This article is to give some ground reality of the challenges in resolving some of the customer reported issues with complex and powerful replication engines like InfoSphere CDC. These time consuming challenges are apart from the development complexities.

Background:
InfoSphere CDC is an application on top of a database. It captures the changes happening in the database via DDL or DML statements via logical logs and appropriately apply on the target database. This model does not intrude the source database as CDC simply reads the logs as it appear. This capability is needed by many customers as their source database will be in production and the production system must not be hindered by any extra load.

Platforms:
InfoSphere CDC is supported on almost all the major platforms, on all the major operating systems and on almost all the major databases. So, a simple problem reported by customer could require a specific environment to be set by the engineering for reproducing the problem in-house and to be able to debug when the reported issue reproduces.

Data:
InfoSphere CDC deals with the data that are usually highly confidential for the customer. This prevents customer to share any such data when there is an issue. Most of the time, it is the data that is to be suspected however, the engineering team does not have visibility of such data. The only option engineering team has is to get the maximum information (clues) from the customer and the traces of CDC and try to generate a data that might cause the same issue seen by the customer. This is fine but the time is the 'fine'!

External influence:
InfoSphere CDC depends heavily on the stability of the underlying database, hardware, operating system and their resources. Since all these are crucial for a successful data replication, CDC can be sensitive when any of these misbehaves or tampered. As you see, these can create the issues that are extremely timing related and very hard to reproduce in order to see the actual issue.

Incorrect configurations:
Also, most of the times, an issue is manifested due to incorrect configuration. It is extremely important for engineering to completely understand the customer environment before they realize that the specific configuration is inappropriate for their environment.

Operational issues:
Like any other softwares, InfoSphere CDC is also affected when its proprietary data is tampered externally. Also, any tampering of database, hardware, operating system and their resources are likely to showup in CDC. Strictly following the best practices helps to dilute these problems.


Comments

  1. Manju16 July 2012 at 01:07

    I am looking for candidates for IBM Info sphere CDC experience. can you let me know what skills should I look for. these candidates are developer/ admin/testers?

    ReplyDelete

Post a Comment

Popular posts from this blog

Informix 14.10 - EAR - Encryption At Rest (Dbspaces and Backups) with multiple levels of ciphers

Disk Encryption - Operational Data Only Up to IDS 12.10 the encryption was allowed only on the operational data on the dbspaces that the server actively performing read/write operations. The Backup data are not encrypted although there is a provision to give control to the end-user to do it via BACKUP_FILTER and RESTORE_FILTER . IDS does not control any encryption that is done with this method and hence takes no responsibility if the encryption key is lost or due to some other reason the decryption becomes impossible. In order to encrypt the operational data, all needed is to configure the onconfig parameter DISK_ENCRYPTION . It uses keystore file ($INFORMIXDIR/etc/*.p12) and stash file ($INFORMIXDIR/etc/*.sth) to support the encryption. These files are created automatically in 12.10 when DISK_ENCRYPTION is configured. The keystore file is used to store the encryption key and stash file contains the password to open the keystore file where encryption key is stored.  Thes...
Read more

InfoSphere CDC (ISCDC) - FAQ

1. What is Latency in Change Data Capture? Latency information is normally for Mirroring subscription. As InfoSphere CDC measures latency as the amount of time that passes between when data changes on a source table and when it changes on the target table. For example, if an application inserts a row into the source table at 10:00 and CDC applies that row to the target able at 10:15, then the latency for the subscription is 15 minutes. 2. What is in-scope and out-of-scope tables in InfoSphere CDC terminology? The tables are inscope tables if they are replicated to the target by InfoSphere CDC. The tables are out-of-scope when those tables are not part of any replication. 3. How to ignore error that occurs during refresh/mirrorring? Prior 6.3 D_MIRROR_REFRESH_ERROR_STOP=OFF - refresh continues after an error has occurred. D_MIRROR_MIRROR_ERROR_STOP=OFF - mirroring continues after an error has occurred. Post 6.3 Set the following 2 system parameters via Management Console. mirror_end...
Read more

HCL Informix 14.10 - onkstore utility - Remote Key Store

Read my blog on Encryption At Rest (EAR) before this article HCL Informix 14.10 has come up with a new utility called onkstore to generate the keystore (.p12) and stash file (.sth) for types such as local AWS_EAR AWS_BAR Type: local This is provided to store the .p12 and .sth files locally and to have our backward compatibilty with earlier versions which were storing these files locally. These files were essential to perform the encryption of the data and eventually decrypt them when needed. No connectivity to the external AWS site is needed to perform this however since the encryption key files are saved locally on the disk, there is a danger of loosing them and hence loosing the encrypted data that remains encrypted for ever and no way to decrypt them Type: AWS_EAR In order to avoid loosing of the encryption key, we have a new type called AWS_EAR to make sure the encryption key is saved safely and remotely so that can be accessed from amazon web-services, w...
Read more