Indeed, where would I be without them? Narvenyl’s new DataSpiders create new possibilities in database abstraction that makes me “geek out”.
The problem is simple. You define what each table looks like, and you can get data from each table in a nice list of objects. However, you can’t minipulate that data, or extend beyond one table, before the results are compiled into an orderly format and returned back to the caller.
From your, the programmer’s, point of view, you ask for purchase #382 from your store’s database, and you get purcahse #382. You know what is inside that object, because you’ve defined what fields are in each object.(For example, the fields “date” and “payment type” are very predictable. However, what about a field called “purchased”? Suddenly, you no longer have a simple, easy-to-store value. Now you have a list of items with their own values!
SQL developers know exactly what I’m talking about. You SELECT data from a database, and then you SELECT more afterwards to get the related data, such as items purchased, addresses used, etc. All this is done largely by hand.
Narvenyl is supposed to automate this kind of task, but how? The fields are defined, and you query those fields and return them. It’s fairly inflexible.
The solution is quite simple. Here’s what a field definition might look like for an e-mail table before my DataSpider idea took hold:
<field name=”id” type=”int” size=”11″ defaultvalue=”” />
<field name=”to” type=”string” size=”255″ defaultvalue=”” />
<field name=”from” type=”string” size=”255″ defaultvalue=”” />
<field name=”heading” type=”string” size=”1024″ defaultvalue=”” />
<field name=”body” type=”string” size=”1024″ defaultvalue=”” />
Now take a look at what it looks like with a DataSpider:
<field name=”id” locale=”field” type=”int” size=”11″ defaultvalue=”” />
<field name=”to” locale=”field” type=”string” size=”255″ defaultvalue=”” />
<field name=”from” locale=”field” type=”string” size=”255″ defaultvalue=”” />
<field name=”heading” locale=”field” type=”string” size=”1024″ defaultvalue=”” />
<field name=”body” locale=”field” type=”string” size=”1024″ defaultvalue=”” />
<field name=”attachment” locale=”join” type=”JoinSpider” size=”1″ defaultvalue=”” joinType=”left” externalTable=”email_attachment_r” condition=”%%this%%.id = %%that%%.email_id” />
There’s quite a few new things going on here. You’ve got locale, which tells the field where it’s actually going to be put in the query. You’ve also got a new field, called attachment, at the very bottom. Note the type: it’s using type “JoinSpider”.
What’s that doing there? Well, the way I’ve decided to do this is, if the type name is actually the name of a pre-defined DataSpider, it’ll actually ask the DataSpider to handle that part of the query. This means I can have DataSpiders handle just about any complex task I put them to. (They might also be able to define new fields, on the fly, that were never spoken of in the original definition. Still have yet to come to a decision on that.)
This allows the DataSpider to manage complex operations that are required to retrieve that particular field from the database. In this case, the DataSpider will retrieve and store any attachment that is related to the current e-mail. It’ll return an array of attachment objects. Because the DataSpider uses the same database object as the one that is calling it to query for the information it needs, any DataSpiders that are defined for attachment objects are called when an attachment is queried! This creates a recursive scenario perfect for easy data retrieval in a tree format.
Problem solved. 🙂