PDA

View Full Version : Datalink and XML



ZachSchuster
04-16-2010, 01:49 PM
Here's a question for people smarter than I am!

In our various athletic venues, our Sports Information crew has a program called StatCrew. This program keeps track of the game scores, play by plays, leaders, etc... it also outputs the data into an XML file. Viewers online can go to a webpage and see all of the games stats live, as the game happens. Here is an example: http://www.cugoldenbears.com/livestats/baseball/xlive.htm

I had planned on having my CG operator copy and paste information (like leader boards) from the webpage and into LiveText. Then I got to looking at Datalink capabilities and began wondering if this could be done automatically.

The LiveStats webpage is not database driven, it's XML... I don't really know what that means. But I'm wondering if anyone is aware of a means by which a person can continuously pull XML data from a file online, then constantly rewrite it into a text file as "keys" and "values" for Datalink.

I'm really in the dark when it comes to XML... if this questions makes no sense, allow me to rephrase this way: I want a way for software to pull data from an XML source and convert it into a txt file for Datalink. :D

Any thoughts, or can someone point me toward an expert in this field?

Thanks,

Zach

jmmultex
04-16-2010, 02:46 PM
Hi Zach,

I am not a database expert by any means but I have a good feel for what would be involved here. XML files are simply text files with data organized into well structured element/value pairs. They are typically easy to read and extract without needing mad programming skills.

If you are able to get an XML file from this site, the best way to process it would be to write a php script (or similar) that looks for the elements you want to display in livetext, pulls out the associated values, and inserts them into a table in a MySQL database. I believe that Datalink can be configured to pull data out of MySQL for formatting under livetext. This script would then need to be run at some reasonable interval that will keep the data current without hammering the site hosting the XML data file. This could be handled by a job scheduler like CRON.

There is clearly more detail here than I outlined, but this is the general flow of what will need to happen. When I get a chance I'll take a look at the link you provided and see if there is anything else I can add to this.

Hope this helps...

-John

jmmultex
04-16-2010, 02:51 PM
BTW - If you simply want a text file for Datalink, this can be even simpler. Ignore all of the database stuff, pull out the element/value parts of the XML file using php, and write them to another text file in a directory that Datalink is monitoring.

This is really just opening one text file (the XML file), and reformatting it into another text file (TXT for Datalink).

John

SBowie
04-16-2010, 02:55 PM
I'm no expert either, but I think XML is a fairly easy format to manipulate, so it should be possible for someone with a little coding skill to work that out for you. DataLink (either standalone for SD TC's or embedded in LT2) can access external data in a number of ways:

In addition to the support for realtime gameclock feeds, it has 'watchers' that can update graphic displays based on either 1) changes to simple ascii text files, or 2) changes in values assigned in a custom xml file that are derieved from MySQL queries.

ZachSchuster
04-16-2010, 05:36 PM
I'll have to find out more from my school's IT folks. I got access to one of the XML files, and it looks like all of the stats are assigned by player. So if I wanted to display the top five leaders in three pointers for basketball, I have no way of sorting the players by three pointers.

Looks like I need some way to take the XML data, put it into a database, then have Datalink query the database to sort the data that I need... pretty much what John said, I think.

jmmultex
04-16-2010, 07:04 PM
The biggest benefit from having the information stored in a database is that you can construct queries that can render all different views into the data - and do it dynamically. The more data that you keep - even data outside of the XML you pull - the more interesting views you can create and display.

Overall, it's a bit more work up front to get it set up this way, but it ultimately let's you do things that aren't possible with fixed text file renders.

I am trying to do something similar to this, but with RSS feeds. If I get the time to get something working on this end, I'll send it to you as a start point for what you want to do.

John

Hopper
04-16-2010, 07:11 PM
First, take 20 minutes or so and get a quick rundown on XML:
http://www.w3schools.com/xml/xml_whatis.asp

It's very simple, and once you get the basics, it will give you a better idea of how you should go about manipulating the XML feed.

In order to make the most of the data, you should find out if the site provides a DTD, or XSD file to go along with the data. DTD and XSD are basically the "format and rules" to the data feed. It defines each element and its attributes. Use this so you will know what to expect from the feed. You can't assume it will be a static structured feed. The elements and their orders can change at any time. That's the beauty of XML.

If your not much on programming, get someone to write you a SOAP interface to the website. From there you can do with the data whatever you like (i.e. PHP, Perl, Java, or just about anything - including a generic text file).

If you can tell me exactly what system you plan on using to read the website data with and what you have available, I can probably give you a good idea of what to expect. I can probably recommend some easy tools for you to use so you can get this done - or write you one.. one of the two.

You also need to find out if they provide this as standard output, or if they provide the feed as a "Web Service" off of another port. This would be similar to an RSS feed, but with slightly different rules.

ZachSchuster
04-16-2010, 07:39 PM
Hmmm... lots of things to consider with this. I'll let you know what I find out.

StatCrew has an application called "GameViewer" for each of their sports that pulls the XML data and sorts it in exactly the manners I want. Commentators and such use this application to see all the game's stats. You have to map a network drive to the directory with the file, so there are network considerations. (This is cheaper than buying terminal screens to duplicate the data to many displays.)

I'm going to call StatCrew next week. I wonder if they could be convinced to develop a modified GameViewer specifically for LiveText. It basically takes all the statistical information that it normally views, but writes it to a text file as keys and values (ie. homescore = 25, home3ptlead1 = Michael Jordan). Then re-saves that file every 1 second or so.

There would be a LOT of keys, and it would be my responsibility to create pages in LiveText that display the information as desired. But heck, if I can get access to the real time data in a format that LT can use, I am perfectly fine importing the fields.

I bet a lot of schools would take advantage of this... live CG updates for the types of stats you see on broadcast television? Big incentive to use LiveText, and big incentive to use StatCrew.

joseburgos
04-17-2010, 12:46 PM
Zach,
The Statcrew has a RS232 program called TasComms32 which can be used to send serial data from the stat computer to a scoreboard and even has a Daktronics mode for the data. In theory, this may already be all you need do and from what I read on the web site, it seems like a free download to try. The PDF manual on it seems to spell everything out as far as config goes.
That all said, I think your idea to contact Statcrew is possibly the best way to go about all this as it is in their best interest to make the interface via network or serial interface.

Good luck,

ZachSchuster
04-17-2010, 02:40 PM
Jose,

The RS-232 option will not work for many schools (mine included), for the very same reason Datalink does not work for many schools. In many press boxes, the RS-232 ports between the stat computer and the scoreboard controller are used to communicate with each other in the DSTI protocol.

Last year I urged Newtek to talk to Daktronics (even referring them to a specific engineer) to discuss the best way to get scoreboard controller data in to Datalink. The RS-232 method is not it, as in many venues it is already used as I explained above, and a lot of controllers don't even have an RS-232 port! See my thread from a while back: http://www.newtek.com/forums/showthread.php?t=101885

It sounds like Newtek is working on this, but as you can see in that thread, they don't comment on projects under development.

What would be REALLY great is a box that plugs into both the scoreboard controller AND the stats computer and sends the combined information to a computer with Datalink wired or wireless.

But for now I think I'll have to go with contacting StatCrew to see if they'd be interested in developing a derivative of their GameViewer.

Here is to hoping!

joseburgos
04-18-2010, 08:20 AM
I see...
Well one thing to take into account is that NewTek not able to comment at that time may be because of LiveText 2 with built-in DataLink. Maybe they made improvements to the "hooks" it has to external devices and data/XML. I plan on up grading to LiveText 2 and will love the Data ability since I presently do not own DataLink.
As far as StatCrew, I think you have the best approach with asking them to make it seamless. I can see it being very good for them if they did.

Take care,

animlab
04-19-2010, 08:59 AM
If you can send me a sample data, protocols ..etc,. and specified what data you want to display, I can come out a program for your system in a few days. Please check my web link for a example:
http://www.animlab.com/NTAddons/eng/index.html
please click the LiveText/LTMate link on the left.

SportsJunkie
04-19-2010, 07:05 PM
Generally, StatCrew would take the stats and write them out to a template file which then appears on the Video Board - not the Scoreboard. Unless you are using a full matrix video board as both your scoreboard and your instant replay/advertising board/messaging board, then your stadium will have two separate pieces of gear: a matrix video board mounted on top of a scoreboard. These are two different animals, controlled by two different controllers.

As an example, on the TransLux (Fair-Play) boards, the MP72 scoreboard controller manipulates the scoreboard. The Proline 900 computer manipulates the matrix video board. On the Daktronics, the Allsport 5000 scoreboard controller manipulates the scoreboard and the Venus 7000 computer manipulates the matrix video board. StatCrew talks to the video board, not the scoreboard.

StatCrew has about a bazillion different reports from which to choose. IMHO, you can probably find a pre-formatted report that will display the stats that either your fans or you might be interested in and you won't need to try and re-import the data back into a MySQL database. Most of the StatCrew packages not only store stats for today's game, but also the entire season. Some versions can keep track of the entire league.

The StatCrew com-port option can be used to either "drive" a remote terminal (VT-100) or to ftp a report to a remote computer over the internet. Pretty neat if you start thinking about the numerous ways someone could log the data (in StatCrew) and have the data wirelessly write to a subdirectory that DataLink can parse. (For instance, the coaches keep the stats for you and they don't have to know anything about what you're doing with the data on the back-end. They don't want you in the coaches box anyway.)

What I am describing above has absolutely nothing to do with "pulling" the data off of the scoreboard controller and inserting it into Live Text. It is simply an example of one way StatCrew exports data from its own application.

animlab
04-20-2010, 09:31 AM
Thank you, I learn a lot.
We don't have any scoreboard and something StatCrew here in Taiwan since sports are not a big market here. :cry:

Cabe
05-04-2010, 05:03 PM
Hey at least you guys have score boards, I gotta parse server logs to get the information from our computer gaming events (and Xbox360's don't even have servers!)

"You, graphics monkey, type faster!"

stijef
07-01-2011, 03:15 AM
One question, important one: can DataLink use .xml file that is on the local network? I have xml file produced on other computer and I'm wondering how to set up address under DataLink's "Network Feed Settings"?

Don't know will this "file://" protocol work if I type the address on some other way or it _must_ be only http protocol? However: when I type in this address "file://192.xxx.xxx.xx/Users/User 1/Desktop/rukomet_t3/xml/ebhc.xml" LiveText can't see the file.

HELP? Every feedback will be useful. Thank you!

brwainer
07-01-2011, 05:39 AM
You could try mounting the network folder as a local drive, that way your operating system takes care of the network issues takes care of the network stuff. Directions vary by operating system, what are you running (Windows or Mac) and what version?

stijef
07-01-2011, 06:08 AM
Sounds interesting... do you have experience with that brwainer from some earlier project, how you did that? (I'm on Windows OS)

I found the way by starting IIS server on that computer that holds XML (root directory of one site is the directory with the xml file). And that way I can aproach to that PC and file by the address http://198.xxx.xxx.xx/file.xml

brwainer
07-01-2011, 08:01 AM
I haven't used Datalink myself, but mounting a remote location as a local drive is the general go-to solution when remote (network) files are visible in Finder or Explorer, but not in a particular application.

Enabling server support is simply a way to enable file sharing, which I had assumed you had already taken care of (you know what they say, "ASSumptions make an A** out of you and me"... or in this case, me :) ). My solution was for if the file was visible in Explorer (on the computer running Datalink) but not in Datalink itself, which is what I took your original message to mean.

stijef
07-02-2011, 12:49 AM
I haven't used Datalink myself, but mounting a remote location as a local drive is the general go-to solution when remote (network) files are visible in Finder or Explorer, but not in a particular application.

Enabling server support is simply a way to enable file sharing, which I had assumed you had already taken care of (you know what they say, "ASSumptions make an A** out of you and me"... or in this case, me :) ). My solution was for if the file was visible in Explorer (on the computer running Datalink) but not in Datalink itself, which is what I took your original message to mean.

Sounds logical and sounds as a solution that can work. I'll try that for sure... but maybe only after this multi-day project. Now, after all the problems with networking, I'll just leave this as it is until the end of the project ("don't change winning crew" ;-))

ASD20
01-05-2012, 10:32 AM
Has anyone figured out how to take the XML from StatCrew and transformed it into a database so Datalink can access the information live during a game?

We are hoping to add this to our broadcasts, since we run StatCrew and use Datalink to pull the scoreboard feed from an AllSportCG.

Thanks for your help!

ZachSchuster
01-05-2012, 10:55 AM
Has anyone figured out how to take the XML from StatCrew and transformed it into a database so Datalink can access the information live during a game?

We are hoping to add this to our broadcasts, since we run StatCrew and use Datalink to pull the scoreboard feed from an AllSportCG.

Thanks for your help!

I had some success using Excel, but not enough to make it worth continuing. I have had better success getting Excel to work by reading the StatCrew HTML pages, but those pages are not as robust as the XML file (as far as what data is available).

Also, StatCrew is releasing new software next year that is going to change a lot of things for every sport. I'm not sure what changes will be made to the XML file, but I'm hesitant to devote time in to making it work if I might need to reinvent the wheel next fall.

There are a couple threads already discussing this topic, too. Those might shed more light on the issue.

sergimari
01-15-2013, 07:16 AM
I was just having the same problem stijef was mentioning, and will test the IIS solution right now, but what about XML interpreting?
Can Datalink interpret the contents and fields of an XML straightforward?
I have been able to load into datalink+livetext the fields from an RSS, but not yet of an XML file. Looks like it should be pretty much the same for both cases, but for whatever reason it is not.

Any help would be greatly appreciated.

stijef
01-15-2013, 07:42 AM
I think it's about XML structure; fields in XML (RSS is nothing than XML File) must be composed on precise, exactly way as LiveText expect. If not, LT will behave as connection to the file is not valid (none from the data will be available through right-click menu).

sergimari
01-15-2013, 08:46 AM
Thanks, but do you know which is the exact way or structure that Livetext expects in an XML file?

I am trying to make it read fields like:

<Field no="656"> 44 </Field>

But I think it will only read fields like:

<Field656> 44 </Field656>

Am I right? Hopefully not...

I am trying to make it work with the XML output from the Total Cricket Scorer 6.0 software. It's a live scoring software so I can't really modify the xml it outputs while I am doing the graphics.

stijef
01-15-2013, 09:26 AM
I'm afraid LiveText can't read attribute value, but just node value. From what I know and from all my experience, it will only see 44, not value of "no". Unfortunately...

If I understood well the application create XML file on some folder, and some for you important data is stored as attribute value; right? If yes I think I can reorganize XML in order to LT see those data too... skype me or drop me an email if you can't solve it different way.