Heterogeneous Data *Target*

I want to create a custom read-only pull subscriber to populate a non-RDBMS
database, for which I can't use any of the existing subscriber types (e.g.
SQL, ODBC/OLE).
I understand enough about how the replication process and tables work (SQL2K
at the moment) to do this. So, for example, I can do the following:
1. Set up a local (i.e. same SQL Server as publisher) SQL subscriber to
replicate the data set I want into a database on the same SQL Server. E.g.
replicate from myserver.sourcedb to myserver.targetdb. This works well; i.e.
I can make updates in sourcedb and see them propagate through to targetdb,
and when sp_MSdistribution_cleanup runs it cleans up
distribution..msrepl_commands correctly.
2. I can then disable / remove the Replication Distribution job, and
commands will begin to queue up in distribution..msrepl_commands.
3. I can then service these commands myself using my custom distributor,
parsing the commands to update my target database. The rest of the process
remains the same; i.e. the log reader and the standard cleanups, so I'm only
replacing the Replication Distribution process / job.
My questions are:
1. As this specific activity is (apparently) undocumented, can I rely on MS
continuing to use a similar model for replication in future SQL Server
versions? I suppose we can tell this for 2005 already although I don't know
the answer to that yet. I wouldn't find a small amount of change a problem
as long as the basic model remained the same.
2. I am parsing the SQL commands in msrepl_commands.command. All the
commands I've seen so far are 'simple'; i.e. they are of only a handful of
forms such as:
'update <table> set <column> = <value>'
, or
'delete <table> where <primary key col 1> = <value 1> and <primary key col
2> = <value 2>'
, etc.
Then there are a few snapshot-related commands which I can recognize.
Again, is this something I can rely on? This may not be worth the effort if
it transpires a sophisticated parsing function is actually required. Note: I
turn off SP replication and ignore schema changes for now.
3. Can I be reasonably sure that some routine or ad-hoc maintenance process
will not - as a goodwill gesture - 'fix' the replication for me and reinstate
the standard Replication Distribution job?
Thanks.
this is not a good path to go down. You are making yourself vulnerable to
the next sp, patch or version of SQL Server MS releases. Also if things go
south you cannot rely on MS for support. Also you are locking your company
into a solution which only you understand.
This being said, you should be safe for the immediate future. SQL 2005 is
similar to SQL 2000 tranny repl.
I take it you are using sp_browsereplcmds to correctly format everything for
you. You can also use sp_dumpparam (I think that's the name of the command)
for more info.
Hilary
"sellotape" <sellotape@.discussions.microsoft.com> wrote in message
news:3116B0C9-967D-48A4-9B65-60394E33E181@.microsoft.com...
>I want to create a custom read-only pull subscriber to populate a non-RDBMS
> database, for which I can't use any of the existing subscriber types (e.g.
> SQL, ODBC/OLE).
> I understand enough about how the replication process and tables work
> (SQL2K
> at the moment) to do this. So, for example, I can do the following:
> 1. Set up a local (i.e. same SQL Server as publisher) SQL subscriber to
> replicate the data set I want into a database on the same SQL Server.
> E.g.
> replicate from myserver.sourcedb to myserver.targetdb. This works well;
> i.e.
> I can make updates in sourcedb and see them propagate through to targetdb,
> and when sp_MSdistribution_cleanup runs it cleans up
> distribution..msrepl_commands correctly.
> 2. I can then disable / remove the Replication Distribution job, and
> commands will begin to queue up in distribution..msrepl_commands.
> 3. I can then service these commands myself using my custom distributor,
> parsing the commands to update my target database. The rest of the
> process
> remains the same; i.e. the log reader and the standard cleanups, so I'm
> only
> replacing the Replication Distribution process / job.
> My questions are:
> 1. As this specific activity is (apparently) undocumented, can I rely on
> MS
> continuing to use a similar model for replication in future SQL Server
> versions? I suppose we can tell this for 2005 already although I don't
> know
> the answer to that yet. I wouldn't find a small amount of change a
> problem
> as long as the basic model remained the same.
> 2. I am parsing the SQL commands in msrepl_commands.command. All the
> commands I've seen so far are 'simple'; i.e. they are of only a handful of
> forms such as:
> 'update <table> set <column> = <value>'
> , or
> 'delete <table> where <primary key col 1> = <value 1> and <primary key
> col
> 2> = <value 2>'
> , etc.
> Then there are a few snapshot-related commands which I can recognize.
> Again, is this something I can rely on? This may not be worth the effort
> if
> it transpires a sophisticated parsing function is actually required.
> Note: I
> turn off SP replication and ignore schema changes for now.
> 3. Can I be reasonably sure that some routine or ad-hoc maintenance
> process
> will not - as a goodwill gesture - 'fix' the replication for me and
> reinstate
> the standard Replication Distribution job?
> Thanks.
|||"Hilary Cotter" wrote:

> this is not a good path to go down. You are making yourself vulnerable to
> the next sp, patch or version of SQL Server MS releases. Also if things go
> south you cannot rely on MS for support. Also you are locking your company
> into a solution which only you understand.
Thanks; that's the answer I was expecting.
I was wondering something that I didn't ask explicitly in the original
question. This is a common requirement (in general); namely to replicate a
part of a data source to another repository. It's not always the case that
the target is a compatible RDBMS. Some companies replicate parts of their
personnel data (via LDAP) to ADS (or other DS) trees, others replicate to
authentication or access systems, address lists and so on. What we're all
doing in the absence of a fairly generic solution like I'm suggesting is
probably a combination of *additional* triggers, *additional* extended SPs,
polling (more highly latent and possibly less efficient) for changes, or
taking full snapshots (again more highly latent and possibly less efficient).
Mostly, we are either taking a latency and / or efficiency hit, or we are
just re-inventing the existing replication metaphor. Usually we are also
unnecessarily modifying the data source (additional triggers, extended procs,
timestamps) to achieve this.
As MS already provides a documented solution for replication into MSSQL from
heterogeneous sources; is it not a little limiting that it does not provide
one for replication out to heterogeneous targets? Then again perhaps it
does, but I've not spotted it yet.
Then just as a sidebar, in this context it's perhaps interesting to note
that even when sticking only to fully vendor-supported solutions this is no
guarantee that additional work will not be required when upgrading. E.g.
several VC++ / VB upgrades have not been backwardly compatible and required a
fair amount of manual rework. I'm not saying it's not understandable; just
that it happens.

HELP: Merge Agent failed - Error number: MSSQL_REPL-2147199402)

We have 50+ clients that use a pull merge subscription. There are 2
databases and each databases has only 1 publication. Each client has ONE
subscription to each of the 2 databases.
We had to set the retention to unlimited to avoid the following error:
The Merge Agent failed after detecting that retention-based metadata cleanup
has deleted metadata at the Publisher for changes not yet sent to the
Subscriber. You must reinitialize the subscription (without upload).
(Source: MSSQL_REPL, Error number: MSSQL_REPL-2147199402)
Get help: http://help/MSSQL_REPL-2147199402
The article http://support.microsoft.com/kb/919929/kb/919929 has a hotfix
BUT it says that it is if a Subscriber subscribes to two publications in the
same Subscriber database. This is not the case.
We had a ticket open with Miscrosoft but they were unable to find the
solution for it after 4 or 5 weeks.
The clients use their subscription databases directly and they are synced to
the server every 30 minutes.
Anybody have any idea what is causing this and how to solve this problem.
The LDF's are getting quite large and we need to find a solution soon.
Thanks in advance!
I fail to understand how the ldf's fit in here. It sounds like there are
some long running transactions which transcend the log dump boundaries. Have
you investigated what these open transactions are? Are they replication
related?
Hilary Cotter
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com
"RichardD" <RichardD@.discussions.microsoft.com> wrote in message
news:B0445940-5C26-403B-8589-692458B82098@.microsoft.com...
> We have 50+ clients that use a pull merge subscription. There are 2
> databases and each databases has only 1 publication. Each client has ONE
> subscription to each of the 2 databases.
> We had to set the retention to unlimited to avoid the following error:
> The Merge Agent failed after detecting that retention-based metadata
> cleanup
> has deleted metadata at the Publisher for changes not yet sent to the
> Subscriber. You must reinitialize the subscription (without upload).
> (Source: MSSQL_REPL, Error number: MSSQL_REPL-2147199402)
> Get help: http://help/MSSQL_REPL-2147199402
> The article http://support.microsoft.com/kb/919929/kb/919929 has a hotfix
> BUT it says that it is if a Subscriber subscribes to two publications in
> the
> same Subscriber database. This is not the case.
> We had a ticket open with Miscrosoft but they were unable to find the
> solution for it after 4 or 5 weeks.
> The clients use their subscription databases directly and they are synced
> to
> the server every 30 minutes.
> Anybody have any idea what is causing this and how to solve this problem.
> The LDF's are getting quite large and we need to find a solution soon.
> Thanks in advance!
>
|||I also expect, the size of the log file has nothing to do with your
retention period.
I'm afraid I can't help on your retention period problem, but perhaps
your ldfs are growing because of a combination of the recovery model
your database is using, and your backup strategy.
Have a look at the following articles, perhaps they will help:
http://msdn2.microsoft.com/en-us/library/ms175987.aspx
http://msdn2.microsoft.com/en-us/library/ms175495.aspx
Tim Hill
On Feb 23, 6:24 pm, "Hilary Cotter" <hilary.cot...@.gmail.com> wrote:[vbcol=seagreen]
> I fail to understand how the ldf's fit in here. It sounds like there are
> some long running transactions which transcend the log dump boundaries. Have
> you investigated what these open transactions are? Are they replication
> related?
> --
> Hilary Cotter
> Looking for a SQL Server replication book?http://www.nwsu.com/0974973602.html
> Looking for a FAQ on Indexing Services/SQL FTShttp://www.indexserverfaq.com
> "RichardD" <Richa...@.discussions.microsoft.com> wrote in message
> news:B0445940-5C26-403B-8589-692458B82098@.microsoft.com...
>
>
>
>
|||You are right. I have already fixed the issue with the log files.
Regarding the error MSSQL_REPL-2147199402 we suspect that the subscribers
changed their computer date beyond 14 days (default retention) and it caused
this error.
One reason we suspect this is that out of 40+ subscribers to the one
publication, only a few had this problem and a couple of them had it happen
twice.
Does this sound like a possibility to you?
We are about to change their retention from unlimited back to 14 days. Is
there anything we should be aware of before doing this?
Thanks so much for your help.
|||Yes, we figured out the log file issue. Thank you anyway.
We are still looking for the solution for the Error number:
MSSQL_REPL-2147199402.
Any ideas?