Dear Dana
You have the ability to either flow JSON or XML as (currently) byte arrays or strings and use C# libraries such as NewtonSoft, System.XML or your own ⺠to process them. Or you can use an XML or JSON extractor (we currently have one of each published as samples on the U-SQL github repo), to extract parts of a JSON or XML document and map parts onto rows (some columns in turns can be maps or arrays).
So it is not taking the route of XQuery or JSONiq in terms of processing arbitrary nested trees without the need of transformations (yet).
To answer your questions inline:
U-SQL is a query language more than a scripting language in that it does not provide snapshots or states inside a U-SQL âscriptâ. However, the syntax of step-wise refinement and composition is closer to a scripting language experience than for example the equivalent common-table-expressions that you normally see in ANSI SQL dialects.
And yes you are right regarding the composition. I would recommend the following blog post to give you some more background about the language philosophy: http://blogs.msdn.com/b/visualstudio/archive/2015/09/28/introducing-u-sql.aspx. It is and evolution of SCOPE (see link in blog post).
Cheers
Michael
From: daniela florescu [mailto:***@me.com]
Sent: Wednesday, November 4, 2015 12:39 AM
To: Michael Rys <***@microsoft.com>
Cc: talk <***@x-query.com>
Subject: Re: [xquery-talk] specification for Microsoft's U-SQL ?
Importance: High
Dear Michael,
I still have a question: can we use U-SQL to process JSON or XML?
I couldât see a way of doing this â as they are not flat rowsâŠ
So is it possible ?
Thanks, regards
Dana
On Nov 3, 2015, at 10:23 PM, daniela florescu <***@me.com<mailto:***@me.com>> wrote:
Dear Michael,
Thanks for the pointers.
I was looking at https://azure.microsoft.com/en-us/documentation/articles/data-lake-analytics-u-sql-get-started/<https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fazure.microsoft.com%2fen-us%2fdocumentation%2farticles%2fdata-lake-analytics-u-sql-get-started%2f&data=01%7c01%7cmrys%40microsoft.com%7ca3ac8bc2c0994a17685008d2e4f374fe%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=XEqoDPUPCck9Jhw5nz1ZVeQMsCLYX3X4iu8t92z%2fLYQ%3d>
and trying to understand the big picture from examples.
So U-SQL is a dataflow scripting language, where the basic data model is a set of rows with C# types, and each
step in the data flow is an expression that can mix (at many levels) SQLâs Select-from-where constructs with C# code.
Thatâs what I understand at least. Let me know if I got it wrong.
Seems very useful for expressing complex data transformations of all kinds, and it is still optimizable.
Much more useful for âdata scienceâ (whatever this big word means..) then the series of SQL-wannabes languages
from the NoSQL vendors.
Thanks, best regards
Dana
On Nov 2, 2015, at 4:56 PM, Michael Rys <***@microsoft.com<mailto:***@microsoft.com>> wrote:
No formal specification. Just a language reference that is still under development :). See http://aka.ms/usql_reference<https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2faka.ms%2fusql_reference&data=01%7c01%7cmrys%40microsoft.com%7ca3ac8bc2c0994a17685008d2e4f374fe%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=2XNRKam46acNQGL4gcPr2JEnrmsJUPVB7eigzd61XMQ%3d> (and pardon the format, I am writing it and our doc people "convert" it).
Let me know if you want a personal demo :).
Cheers
Michael
-----Original Message-----
From: talk-***@x-query.com<mailto:talk-***@x-query.com> [mailto:talk-***@x-query.com] On Behalf Of daniela florescu
Sent: Saturday, October 31, 2015 2:05 PM
To: talk <***@x-query.com<mailto:***@x-query.com>>
Subject: [xquery-talk] specification for Microsoft's U-SQL ?
Seems interesting in principle.
https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fthenewstack.io%2fmeet-u-sql-microsofts-new-language-big-data%2f&data=01%7c01%7cmrys%40microsoft.com%7caef4ceeec92543784b5608d2e23771f1%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=iZjmF47tPZ316GAyjeXegMSLOq%2bLFAbo3zvRYOwMPy0%3d
But anyone in a luck to see a specification for it ?
Thanks for any pointer, best regards
Dana
_______________________________________________
***@x-query.com<mailto:***@x-query.com>
https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fx-query.com%2fmailman%2flistinfo%2ftalk&data=01%7c01%7cmrys%40microsoft.com%7caef4ceeec92543784b5608d2e23771f1%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=dJTn2RgYiSUgZH5GU3e3CsLqpCj9Gvd6uI%2fGS7JomP8%3d
_______________________________________________
***@x-query.com<mailto:***@x-query.com>
http://x-query.com/mailman/listinfo/talk