Discussion:
[xquery-talk] MarkLogic using JSONiq for processing JSON ?
daniela florescu
2015-04-24 00:24:45 UTC
Permalink
I heard that MarkLogic will be using JSONiq for processing JSON.
http://www.jsoniq.org

Sounds like wonderful news to me.

Hope it’s true.

Best
Dana



_______________________________________________
***@x-query.com
http://x
daniela florescu
2015-04-28 17:15:40 UTC
Permalink
Dear Kurt (Cagle)

on linkedin to the same answer bellow you answered me:

"As I said, first glance says it's close, but I'm not completely up to date on the JSONIQ spec. I know when I talked to a key developer of mutual acquaintance, he indicated that he followed JSONIQ, but that was still while it was under development."


Kurt, do you have now a better idea about the technical differences between the two JSOn query languages: JSONiq designed and supported by Zorba
and the one supported by MarkLogic ?

Or does someone else ?

What is the technical rationale for making the two languages different ? Any strong technical reasons ?

If there are no strong technical reasons, and the two are different just for the sake of being different, that's very sad.

Relational databases survived for 30 years because those guys were brilliant business people and
understood the power of a standard/common language and common APIs for all vendors.

It strengthens the (entire) community to the point that, even 30 years later, it is almost impossible to get SQL out of their hands....

It's very unfortunate that the NoSQL community, and especially MarkLogic who considers themselves the "leaders" in this market,
don't get that simple fact....and they had to twist JSONiq here and there in order to avoid admitting they use the language designed by the
Zorba community and avoid calling it JSONiq....

Such a lack of vision is sad.

But I digress.

I am still curious if someone compiled a list of technical differences.

Thanks, best regards
Dana
Post by daniela florescu
I heard that MarkLogic will be using JSONiq for processing JSON.
http://www.jsoniq.org
Sounds like wonderful news to me.
Hope it’s true.
Best
Dana
daniela florescu
2015-05-03 06:02:18 UTC
Permalink
Dear Gary Bloom,

you are the CEO of MarkLogic, and you are in cc of this message to those two esteemed XML-related mailing lists: xquery-talk and xml-dev.

To be honest, given the marketing screaming that Marklogic did recently about your JSON support (geez, must have been expensive !!!), plus my messages
on those esteemed mailing lists, plus the amount of time and efforts that me and my Zorba team put into designing JSONiq (3 years)

http://www.jsoniq.org


...


...I was obviously slightly disappointed to see that nobody from MarkLogic had the intellectual honesty to give ME (and the others on the mailing lists)
one of the possible answers about MarkLogic’s JSON support:

(a) yes, we, MarkLogic, we inspired ourselves and adapt as much as we could from JSONiq, but we had no balls to admit it, and give proper scientific references to JSONiq

(b) no, we. MarkLogic, we designed everything from scratch without looking at JSONiq (but then, I would ask
 Why !?
.just curious...)

(c) yes, the language we, MarkLogic, support is exactly the same language as JSONiq, but again, we lack the balls to admit it in front of the wholeXML/JSON industry.

or any other such variation of such answer
.. pick your choice.

=============


I am asking you personally, Gary Bloom, CEO of MarkLogic, which one it is !?

Do you have any idea !?

No shame in stealing, Gary. After all, you come from Oracle, and Oracle was built on Ellison stealing IBM’s research, and Larry Ellison just bought an island
.
while IBM isn’t doing great
..


 so I wouldn’t blame you if your plan is the same (small detail, I don’t have the kindness of character that Don Chamberlin from IBM had
just keep this in mind
for your future plans, though
..)

=============

Looking forward to your answer, Gary Bloom.

And in the meantime, yes, I admit, I was lazy. I could have gotten my own answer to that question about the technical differences between MarkLogic support of JSON
and JSONiq (except that was looking forward to some intellectual honesty from MarkLogic, in vain, alas)

I will do my own own investigation about the technical differences between the two, and write a report.

Thanks for your (very telling, I am afraid
.) silence.

MarkLogic has no scientific strength, nor political and economical “balls" (and sorry, I am afraid that’s a technical term).

Best regards
Dana

P.S,. I’ll teach you a very useful Romanian saying : “If you want to kill a cat, you can always say she has rabies”.
Yep, tell anyone you want that I have rabies :-))))))
Post by daniela florescu
Dear Kurt (Cagle)
"As I said, first glance says it's close, but I'm not completely up to date on the JSONIQ spec. I know when I talked to a key developer of mutual acquaintance, he indicated that he followed JSONIQ, but that was still while it was under development."
Kurt, do you have now a better idea about the technical differences between the two JSOn query languages: JSONiq designed and supported by Zorba
and the one supported by MarkLogic ?
Or does someone else ?
What is the technical rationale for making the two languages different ? Any strong technical reasons ?
If there are no strong technical reasons, and the two are different just for the sake of being different, that's very sad.
Relational databases survived for 30 years because those guys were brilliant business people and
understood the power of a standard/common language and common APIs for all vendors.
It strengthens the (entire) community to the point that, even 30 years later, it is almost impossible to get SQL out of their hands....
It's very unfortunate that the NoSQL community, and especially MarkLogic who considers themselves the "leaders" in this market,
don't get that simple fact....and they had to twist JSONiq here and there in order to avoid admitting they use the language designed by the
Zorba community and avoid calling it JSONiq....
Such a lack of vision is sad.
But I digress.
I am still curious if someone compiled a list of technical differences.
Thanks, best regards
Dana
Post by daniela florescu
I heard that MarkLogic will be using JSONiq for processing JSON.
http://www.jsoniq.org <http://www.jsoniq.org/>
Sounds like wonderful news to me.
Hope it’s true.
Best
Dana
_______________________________________________
http://x-query.com/mailman/listinfo/talk
Michael Kay
2015-05-07 09:36:17 UTC
Permalink
I don’t know anything about the MarkLogic implementation, but some of the key differences between XQuery 3.1 and the JSONiq proposal are:

At the data model level:

* Maps can use any atomic value as the key, it does not have to be a string
* The members of an array are sequences, not necessarily items
* JSON’s null is represented as an empty sequence, not as a new atomic data type

At the syntax level:

* In XQ3.1, Map constructors use the syntax map{ a:b, c:d } rather than bare curlies
* XQ 3.1 introduces a lookup operator for maps and arrays: employee?name, or book?author?1

There are many differences of detail, for example XQ3.1 allows an array to be atomized, so that sum() over an array does "the right thing”.

Of course these differences were all very hotly debated over a long period of time, and I wouldn’t even attempt to summarize the arguments.

Michael Kay
Saxonica
***@saxonica.com
+44 (0) 118 946 5893
Post by daniela florescu
Dear Kurt (Cagle)
"As I said, first glance says it's close, but I'm not completely up to date on the JSONIQ spec. I know when I talked to a key developer of mutual acquaintance, he indicated that he followed JSONIQ, but that was still while it was under development."
Kurt, do you have now a better idea about the technical differences between the two JSOn query languages: JSONiq designed and supported by Zorba
and the one supported by MarkLogic ?
Or does someone else ?
What is the technical rationale for making the two languages different ? Any strong technical reasons ?
If there are no strong technical reasons, and the two are different just for the sake of being different, that's very sad.
Relational databases survived for 30 years because those guys were brilliant business people and
understood the power of a standard/common language and common APIs for all vendors.
It strengthens the (entire) community to the point that, even 30 years later, it is almost impossible to get SQL out of their hands....
It's very unfortunate that the NoSQL community, and especially MarkLogic who considers themselves the "leaders" in this market,
don't get that simple fact....and they had to twist JSONiq here and there in order to avoid admitting they use the language designed by the
Zorba community and avoid calling it JSONiq....
Such a lack of vision is sad.
But I digress.
I am still curious if someone compiled a list of technical differences.
Thanks, best regards
Dana
Post by daniela florescu
I heard that MarkLogic will be using JSONiq for processing JSON.
http://www.jsoniq.org <http://www.jsoniq.org/>
Sounds like wonderful news to me.
Hope it’s true.
Best
Dana
_______________________________________________
http://x-query.com/mailman/listinfo/talk
daniela florescu
2015-05-08 09:29:28 UTC
Permalink
Michael,

Nothing seems fundamentally “right” or “wrong” one way or the other in what you say.
(syntax in XQuery 3.1 seems uglier then JSONiq to me though
..)

So, to me, the decisions of the W3C working group seems random, and rather based on a two years old
kind of a tantrum “I WANT TO BE DIFFERETENT JUST BECAUSE
..I WANT IT."




...rather then justified by any technical reasons.


This is why this community will die (unless they change, and this unfortunately involves changing the leaders).


Because everybody thinks about their small itsy-bitsy ego, and their itsy-bisty market, instead of the bigger picture.


D.
Post by Michael Kay
* Maps can use any atomic value as the key, it does not have to be a string
* The members of an array are sequences, not necessarily items
* JSON’s null is represented as an empty sequence, not as a new atomic data type
* In XQ3.1, Map constructors use the syntax map{ a:b, c:d } rather than bare curlies
* XQ 3.1 introduces a lookup operator for maps and arrays: employee?name, or book?author?1
There are many differences of detail, for example XQ3.1 allows an array to be atomized, so that sum() over an array does "the right thing”.
Of course these differences were all very hotly debated over a long period of time, and I wouldn’t even attempt to summarize the arguments.
Michael Kay
Saxonica
+44 (0) 118 946 5893
Post by daniela florescu
Dear Kurt (Cagle)
"As I said, first glance says it's close, but I'm not completely up to date on the JSONIQ spec. I know when I talked to a key developer of mutual acquaintance, he indicated that he followed JSONIQ, but that was still while it was under development."
Kurt, do you have now a better idea about the technical differences between the two JSOn query languages: JSONiq designed and supported by Zorba
and the one supported by MarkLogic ?
Or does someone else ?
What is the technical rationale for making the two languages different ? Any strong technical reasons ?
If there are no strong technical reasons, and the two are different just for the sake of being different, that's very sad.
Relational databases survived for 30 years because those guys were brilliant business people and
understood the power of a standard/common language and common APIs for all vendors.
It strengthens the (entire) community to the point that, even 30 years later, it is almost impossible to get SQL out of their hands....
It's very unfortunate that the NoSQL community, and especially MarkLogic who considers themselves the "leaders" in this market,
don't get that simple fact....and they had to twist JSONiq here and there in order to avoid admitting they use the language designed by the
Zorba community and avoid calling it JSONiq....
Such a lack of vision is sad.
But I digress.
I am still curious if someone compiled a list of technical differences.
Thanks, best regards
Dana
Post by daniela florescu
I heard that MarkLogic will be using JSONiq for processing JSON.
http://www.jsoniq.org <http://www.jsoniq.org/>
Sounds like wonderful news to me.
Hope it’s true.
Best
Dana
_______________________________________________
http://x-query.com/mailman/listinfo/talk
Michael Kay
2015-05-08 11:09:05 UTC
Permalink
Post by daniela florescu
So, to me, the decisions of the W3C working group seems random, and rather based on a two years old
kind of a tantrum “I WANT TO BE DIFFERETENT JUST BECAUSE…..I WANT IT."
……...rather then justified by any technical reasons.
No, all the arguments were all technical. For example:

* generalizing maps to allow any atomic type as the key, rather than only a string, was because of specific use cases that required this (remember that the first proposal to add maps to XDM came from XSLT streaming work, not from JSONiq)

* the decision to use “map{…}” rather than “{…}” was to some extent subjective, but was motivated by technical arguments such as the ability to produce good error messages, retaining options for future extensions to the grammar, etc. Expressions beginning with “{“ are particularly problematic because “{“ is used to delimit embedded expressions in element content, and “{{“ is used to escape “{“ as an ordinary character; they are also very obscure when used as the body of a function (declare function f {{1:2}}). Before making this decision, we looked at how many other popular languages solve this problem.

* the decision to allow any sequence to act as a member of an array enabled things like the fn:apply() function, whose second argument is an array of arbitrary sequences; it also enabled JSON null to be represented by an empty sequence, which avoided the need for pervasive changes to the language to define how every function and operator should handle a JSON null. Reducing the number of concepts by one is a definite plus.

Getting agreement on all these points was a very lengthy process with much heated argument. Although the decisions made were not always the ones I personally advocated, I think the final language works well. If there’s one aspect I’m still a little unhappy about, it’s the fact that an array behaves like a single item, so for example

let $A := [1,2,3]
return $A[1]

returns [1,2,3]

But that’s there because we tried very hard to find a way to avoid this surprise, and failed: the sequence=item model in XDM is just too deeply embedded.

Michael Kay
Saxonica


_______________________________________________
***@x-query.com
http://x-query.com/mailman/li
Adam Retter
2015-05-08 11:21:58 UTC
Permalink
Post by Michael Kay
No, all the arguments were all technical
Getting agreement on all these points was a very lengthy process with much heated argument. Although the decisions made were not always the ones I personally advocated, I think the final language works well.
Absolutely! Whilst I am more of an occasional frequenter and
contributor to the XQuery WG, having seen the level of collaborative
work and perseverance that that has gone into adding Maps and Arrays
to XQuery, I can say that I am impressed.

From my perspective, it was a difficult process, but everyone worked
hard together and the technical arguments were always foremost. Whilst
working under the constraints of backwards compatibility and creating
a cohesive data model and language, I think the result speaks for
itself. Certainly many of eXist's users are very happy with the new
Maps and Arrays work in XQuery 3.1 and we frequently receive positive
feedback on this.
--
Adam Retter

skype: adam.retter
tweet: adamretter
http://www.adamretter.org.uk
_______________________________________________
***@x-query.com
http://x-query.com/mailman/listinfo/talk
Ghislain Fourny
2015-05-08 12:46:03 UTC
Permalink
Hi,

My understanding of the reason why XQuery 3.1 and JSONiq diverge is
also use-case based. If you are interested in some more background, I
believe there were two drivers all along:

1. Support for NoSQL document store querying.
2. Support for map and array structures in memory.

Both use cases are important, even if their importance varies from
project to project or from company to company.

JSONiq was designed 100% for NoSQL document store querying : it is (on
purpose) optimized for supporting use case 1 and only use case 1.

XQuery 3.1 also covers use case 2. Supporting both use cases and
making compromises between the two was not trivial, which is why there
was a lot of work put into the discussions and the design in the
working group.

I share Adam's feeling 100%. Everybody took the time to patiently
listen to one another, which is one of the aspects of the working
group I valued the most. I enjoyed each meeting we had and learned a
lot from the discussions -- paradoxically, I also had the feeling of
knowing JSONiq better after having them.

To me, one of the main concrete technical differences (if I correctly
remember) is that JSONiq manipulates trees/documents (both with XML
and JSON), so that when you put data into an object, it is copied over
(even if most of the time the optimizer can spare the actual copying).
This eliminates many serialization issues when data gets roundtripped
around. XQuery 3.1 is more data-structure oriented as use case 2
requires retaining the original identity of map and array values,
which theoretically allows graphs. This difference is important and
noticeable when updates and scripting kick in.

Of course, for each set of use cases, standards and interoperability
are always desirable, so I can understand Dana's discomfort. I also
think, though, that it is natural for different languages to emerge if
the use cases differ and, in the broader context of IT, think that it
is wise to pragmatically select technologies based on what one wants
to achieve.

On the bright side also: the data models behind the languages are
similar, and both languages can run on the same virtual machine. Zorba
already supports XQuery 3.0 and JSONiq side by side.

I hope it makes sense.

Kind regards,
Ghislain
Post by Adam Retter
Post by Michael Kay
No, all the arguments were all technical
Getting agreement on all these points was a very lengthy process with much heated argument. Although the decisions made were not always the ones I personally advocated, I think the final language works well.
Absolutely! Whilst I am more of an occasional frequenter and
contributor to the XQuery WG, having seen the level of collaborative
work and perseverance that that has gone into adding Maps and Arrays
to XQuery, I can say that I am impressed.
From my perspective, it was a difficult process, but everyone worked
hard together and the technical arguments were always foremost. Whilst
working under the constraints of backwards compatibility and creating
a cohesive data model and language, I think the result speaks for
itself. Certainly many of eXist's users are very happy with the new
Maps and Arrays work in XQuery 3.1 and we frequently receive positive
feedback on this.
--
Adam Retter
skype: adam.retter
tweet: adamretter
http://www.adamretter.org.uk
_______________________________________________
http://x-query.com/mailman/listinfo/talk
_______________________________________________
***@x-query.com
http://x-query.com/mailman/listinfo/talk
daniela florescu
2015-05-10 04:49:48 UTC
Permalink
Ghislain,

just a simple remark about the hypocrisy and/or naivety and/or lack of technical skills in the design of XQuery 3.1.

You say that , unlike JSONiq, it’s NOT designed as a query language for JSON, and that’s why it fails so short in that respect.

If it is NOT designed as a query language for JSON, why did you bother introducing the notion of the JSON nulll in XQuery 3.1 at all !?
(I mean, why not the null pointer from Java !??? Or some Scala structures !??)

Pick your choose:
- either XQuery 3.1 was intended to query JSON, and then it failed, or
- XQuery 3.1 was NOT intended to query JSON, and then it’s damn weird.


Dana
Post by Ghislain Fourny
Hi,
My understanding of the reason why XQuery 3.1 and JSONiq diverge is
also use-case based. If you are interested in some more background, I
1. Support for NoSQL document store querying.
2. Support for map and array structures in memory.
Both use cases are important, even if their importance varies from
project to project or from company to company.
JSONiq was designed 100% for NoSQL document store querying : it is (on
purpose) optimized for supporting use case 1 and only use case 1.
XQuery 3.1 also covers use case 2. Supporting both use cases and
making compromises between the two was not trivial, which is why there
was a lot of work put into the discussions and the design in the
working group.
I share Adam's feeling 100%. Everybody took the time to patiently
listen to one another, which is one of the aspects of the working
group I valued the most. I enjoyed each meeting we had and learned a
lot from the discussions -- paradoxically, I also had the feeling of
knowing JSONiq better after having them.
To me, one of the main concrete technical differences (if I correctly
remember) is that JSONiq manipulates trees/documents (both with XML
and JSON), so that when you put data into an object, it is copied over
(even if most of the time the optimizer can spare the actual copying).
This eliminates many serialization issues when data gets roundtripped
around. XQuery 3.1 is more data-structure oriented as use case 2
requires retaining the original identity of map and array values,
which theoretically allows graphs. This difference is important and
noticeable when updates and scripting kick in.
Of course, for each set of use cases, standards and interoperability
are always desirable, so I can understand Dana's discomfort. I also
think, though, that it is natural for different languages to emerge if
the use cases differ and, in the broader context of IT, think that it
is wise to pragmatically select technologies based on what one wants
to achieve.
On the bright side also: the data models behind the languages are
similar, and both languages can run on the same virtual machine. Zorba
already supports XQuery 3.0 and JSONiq side by side.
I hope it makes sense.
Kind regards,
Ghislain
Post by Adam Retter
Post by Michael Kay
No, all the arguments were all technical
Getting agreement on all these points was a very lengthy process with much heated argument. Although the decisions made were not always the ones I personally advocated, I think the final language works well.
Absolutely! Whilst I am more of an occasional frequenter and
contributor to the XQuery WG, having seen the level of collaborative
work and perseverance that that has gone into adding Maps and Arrays
to XQuery, I can say that I am impressed.
From my perspective, it was a difficult process, but everyone worked
hard together and the technical arguments were always foremost. Whilst
working under the constraints of backwards compatibility and creating
a cohesive data model and language, I think the result speaks for
itself. Certainly many of eXist's users are very happy with the new
Maps and Arrays work in XQuery 3.1 and we frequently receive positive
feedback on this.
--
Adam Retter
skype: adam.retter
tweet: adamretter
http://www.adamretter.org.uk
_______________________________________________
http://x-query.com/mailman/listinfo/talk
_______________________________________________
http://x-query.com/mailman/listinfo/talk
_______________________________________________
***@x-query.com
http://x
daniela florescu
2015-05-09 23:41:17 UTC
Permalink
Adam, Ghislain,


Com’ on. Stop running around the bush. I hate hypocrisy in technology.
Reading your emails made me feel back in the communist era, with the perfect usage of a perfect wooden language.

Since when “working well together” and “listening to each other” (what a wonderful world !!!!)…..: guaranteed a good technical result !???
(Maybe they should have voted to see if Galileo was right or not …. wait, actually they DID…. OUPS.)

Since when this guaranteed any logical result, and any good, usable tool for an industry !?
(Web services would be the first thing that would jump to my mind as a graceful design of a “nice" community that all listened to each other
… as a nightmare equivalent to XQuery 3.1)

More often then not, such a “nice process” ends up in a horrible technical compromise which is good for no-one, and gets abandoned
by industry VERY, VERY soon.

============


You both prove my point: the design of XQuery 3.1 was done to “help” selfishly and with a very short term vision two-three companies (Saxon, Exist, MarkLogic), and with complete
disregard to the big picture of the needs of querying and processing semi-structured data in the industry (which includes both XML and JSON).

As for the choice between solving:
(a) XSLT maps and arrays (estimated # use cases in the Ks, includes all three eXist, Saxon and MarkLogic) and
(b) querying JSON (downloads of Mongo+Cassandra+Cloudera+Spark+Couch+… in the tens of millions of downloads…)


(let alone the comparison in the total sales number…)

……...very intelligently, XQuery 3.1 decided to totally ignore the millions of use cases of Cassandra+Mongo+Cloudera+…., and “serve” Saxon and eXist,
and antagonize all those millions of use cases that need JSON query processing.

Again.

Great.

Thanks for your “contribution" to the industry.


Dana
Post by Adam Retter
Post by Michael Kay
No, all the arguments were all technical
Getting agreement on all these points was a very lengthy process with much heated argument. Although the decisions made were not always the ones I personally advocated, I think the final language works well.
Absolutely! Whilst I am more of an occasional frequenter and
contributor to the XQuery WG, having seen the level of collaborative
work and perseverance that that has gone into adding Maps and Arrays
to XQuery, I can say that I am impressed.
From my perspective, it was a difficult process, but everyone worked
hard together and the technical arguments were always foremost. Whilst
working under the constraints of backwards compatibility and creating
a cohesive data model and language, I think the result speaks for
itself. Certainly many of eXist's users are very happy with the new
Maps and Arrays work in XQuery 3.1 and we frequently receive positive
feedback on this.
--
Adam Retter
skype: adam.retter
tweet: adamretter
http://www.adamretter.org.uk
_______________________________________________
***@x-query.com
http://x-query.com/mailman/listinfo/
Adam Retter
2015-05-10 00:03:36 UTC
Permalink
Hi Daniela,

I am not sure if I need to, but I feel that I might need to defend
myself from your latest email.

Let me be clear, I personally wanted to participate in the XQuery WG.
I was not asked to do it by anyone in the eXist project and neither
did I do it to advance the eXist project. My sole purpose at the time
for enquiring about how I might join without working for a member
organisation of the W3C was because I felt that open source community
implementations of XQuery were unrepresented within the working group.
I felt that the average user of XQuery, that could not afford an
expensive implementation should also be represented within the WG.

I was lucky to be invited to join the WG and I am thankful for that.
Subsequently, I have paid out of my own personal pocket for every WG
meeting I have attended. I can only attend those in Europe as the
others further afield are beyond my means, and I have not always been
able to attend all in Europe due to cost. I work on an Open Source
project, almost all of the work I do is unfunded and I do it in my
evenings and weekends because it is interesting and I enjoy
contributing to the community. I did email the FLWOR Foundation about
possible funding on more than one occasion in the past but never
received a response from you. I have never been paid in any way for my
WG efforts or reimbursed any of my costs by any organisation (that
includes eXist Solutions).

What I am trying to say, is that I am not part of the WG to achieve
some sort of eXist-db domination plan, or for any sort of
machiavellian scheme. I attend the WG meetings because I genuinely
want to improve the language for its users, and for me that is more
important than what eXist, BaseX, MarkLogic, IBM, Oracle or any other
implementation wants. I also teach several XQuery courses each year
entirely free of charge, and again I fund this out of my own pocket,
my goal as always is to help the community. I think that XQuery is a
beautiful language that brings a great deal of freedom to its users.

I understand that you are unhappy with the design of XQuery 3.1, and I
am sorry to hear that; I know that you were deeply instrumental in the
original design of XQuery and spent many years working on the XQuery
WG.

All the best. Adam.
Post by daniela florescu
Adam, Ghislain,
Com’ on. Stop running around the bush. I hate hypocrisy in technology.
Reading your emails made me feel back in the communist era, with the perfect usage of a perfect wooden language.
Since when “working well together” and “listening to each other” (what a wonderful world !!!!)…..: guaranteed a good technical result !???
(Maybe they should have voted to see if Galileo was right or not …. wait, actually they DID…. OUPS.)
Since when this guaranteed any logical result, and any good, usable tool for an industry !?
(Web services would be the first thing that would jump to my mind as a graceful design of a “nice" community that all listened to each other
… as a nightmare equivalent to XQuery 3.1)
More often then not, such a “nice process” ends up in a horrible technical compromise which is good for no-one, and gets abandoned
by industry VERY, VERY soon.
============
You both prove my point: the design of XQuery 3.1 was done to “help” selfishly and with a very short term vision two-three companies (Saxon, Exist, MarkLogic), and with complete
disregard to the big picture of the needs of querying and processing semi-structured data in the industry (which includes both XML and JSON).
(a) XSLT maps and arrays (estimated # use cases in the Ks, includes all three eXist, Saxon and MarkLogic) and
(b) querying JSON (downloads of Mongo+Cassandra+Cloudera+Spark+Couch+… in the tens of millions of downloads…)
(let alone the comparison in the total sales number…)
……...very intelligently, XQuery 3.1 decided to totally ignore the millions of use cases of Cassandra+Mongo+Cloudera+…., and “serve” Saxon and eXist,
and antagonize all those millions of use cases that need JSON query processing.
Again.
Great.
Thanks for your “contribution" to the industry.
Dana
Post by Adam Retter
Post by Michael Kay
No, all the arguments were all technical
Getting agreement on all these points was a very lengthy process with much heated argument. Although the decisions made were not always the ones I personally advocated, I think the final language works well.
Absolutely! Whilst I am more of an occasional frequenter and
contributor to the XQuery WG, having seen the level of collaborative
work and perseverance that that has gone into adding Maps and Arrays
to XQuery, I can say that I am impressed.
From my perspective, it was a difficult process, but everyone worked
hard together and the technical arguments were always foremost. Whilst
working under the constraints of backwards compatibility and creating
a cohesive data model and language, I think the result speaks for
itself. Certainly many of eXist's users are very happy with the new
Maps and Arrays work in XQuery 3.1 and we frequently receive positive
feedback on this.
--
Adam Retter
skype: adam.retter
tweet: adamretter
http://www.adamretter.org.uk
--
Adam Retter

skype: adam.retter
tweet: adamretter
http://www.adamretter.org.uk

_______________________________________________
***@x-query.com
http://x-query.com/mailman/listinfo/
daniela florescu
2015-05-10 04:42:47 UTC
Permalink
Well, Adam, I can look only at the results of those efforts, and judge by the results, sorry. I’m not a mind reader.

I offered two possible explanations fort such a strange outcome as XQuery 3.1 (it’s not an ostrich, and it’s not
a camel, but kind of looks like both).

I said:

"This can have two possible explanations: pure stupidity and lack of good will.”

Ok. I can add another one: technical and political naivety.

Best regards
Dana
Post by Adam Retter
Hi Daniela,
I am not sure if I need to, but I feel that I might need to defend
myself from your latest email.
Let me be clear, I personally wanted to participate in the XQuery WG.
I was not asked to do it by anyone in the eXist project and neither
did I do it to advance the eXist project. My sole purpose at the time
for enquiring about how I might join without working for a member
organisation of the W3C was because I felt that open source community
implementations of XQuery were unrepresented within the working group.
I felt that the average user of XQuery, that could not afford an
expensive implementation should also be represented within the WG.
I was lucky to be invited to join the WG and I am thankful for that.
Subsequently, I have paid out of my own personal pocket for every WG
meeting I have attended. I can only attend those in Europe as the
others further afield are beyond my means, and I have not always been
able to attend all in Europe due to cost. I work on an Open Source
project, almost all of the work I do is unfunded and I do it in my
evenings and weekends because it is interesting and I enjoy
contributing to the community. I did email the FLWOR Foundation about
possible funding on more than one occasion in the past but never
received a response from you. I have never been paid in any way for my
WG efforts or reimbursed any of my costs by any organisation (that
includes eXist Solutions).
What I am trying to say, is that I am not part of the WG to achieve
some sort of eXist-db domination plan, or for any sort of
machiavellian scheme. I attend the WG meetings because I genuinely
want to improve the language for its users, and for me that is more
important than what eXist, BaseX, MarkLogic, IBM, Oracle or any other
implementation wants. I also teach several XQuery courses each year
entirely free of charge, and again I fund this out of my own pocket,
my goal as always is to help the community. I think that XQuery is a
beautiful language that brings a great deal of freedom to its users.
I understand that you are unhappy with the design of XQuery 3.1, and I
am sorry to hear that; I know that you were deeply instrumental in the
original design of XQuery and spent many years working on the XQuery
WG.
All the best. Adam.
Post by daniela florescu
Adam, Ghislain,
Com’ on. Stop running around the bush. I hate hypocrisy in technology.
Reading your emails made me feel back in the communist era, with the perfect usage of a perfect wooden language.
Since when “working well together” and “listening to each other” (what a wonderful world !!!!)
..: guaranteed a good technical result !???
(Maybe they should have voted to see if Galileo was right or not 
. wait, actually they DID
. OUPS.)
Since when this guaranteed any logical result, and any good, usable tool for an industry !?
(Web services would be the first thing that would jump to my mind as a graceful design of a “nice" community that all listened to each other

 as a nightmare equivalent to XQuery 3.1)
More often then not, such a “nice process” ends up in a horrible technical compromise which is good for no-one, and gets abandoned
by industry VERY, VERY soon.
============
You both prove my point: the design of XQuery 3.1 was done to “help” selfishly and with a very short term vision two-three companies (Saxon, Exist, MarkLogic), and with complete
disregard to the big picture of the needs of querying and processing semi-structured data in the industry (which includes both XML and JSON).
(a) XSLT maps and arrays (estimated # use cases in the Ks, includes all three eXist, Saxon and MarkLogic) and
(b) querying JSON (downloads of Mongo+Cassandra+Cloudera+Spark+Couch+
 in the tens of millions of downloads
)
(let alone the comparison in the total sales number
)


...very intelligently, XQuery 3.1 decided to totally ignore the millions of use cases of Cassandra+Mongo+Cloudera+
., and “serve” Saxon and eXist,
and antagonize all those millions of use cases that need JSON query processing.
Again.
Great.
Thanks for your “contribution" to the industry.
Dana
Post by Adam Retter
Post by Michael Kay
No, all the arguments were all technical
Getting agreement on all these points was a very lengthy process with much heated argument. Although the decisions made were not always the ones I personally advocated, I think the final language works well.
Absolutely! Whilst I am more of an occasional frequenter and
contributor to the XQuery WG, having seen the level of collaborative
work and perseverance that that has gone into adding Maps and Arrays
to XQuery, I can say that I am impressed.
From my perspective, it was a difficult process, but everyone worked
hard together and the technical arguments were always foremost. Whilst
working under the constraints of backwards compatibility and creating
a cohesive data model and language, I think the result speaks for
itself. Certainly many of eXist's users are very happy with the new
Maps and Arrays work in XQuery 3.1 and we frequently receive positive
feedback on this.
--
Adam Retter
skype: adam.retter
tweet: adamretter
http://www.adamretter.org.uk
--
Adam Retter
skype: adam.retter
tweet: adamretter
http://www.adamretter.org.uk <http://www.adamretter.org.uk/>
Ihe Onwuka
2015-05-20 08:35:57 UTC
Permalink
Post by daniela florescu
You both prove my point: the design of XQuery 3.1 was done to “help”
selfishly and with a very short term vision two-three companies (Saxon,
Exist, MarkLogic), and with complete
disregard to the big picture of the needs of querying and processing
semi-structured data in the industry (which includes both XML and JSON).
Ok this is anecdotal and on a somewhat different tack but a couple of
months ago I took the Marklogic Developer course. One of the few things I
retained from the course was the realisation that the instructor had no
idea what was in the XQuery specifiation (1.0 or 3.0) and in what way they
differed from the Marklogic variant.

Now that certainly makes me wonder whether that is reflective of the
corporate technical viewpoint.
daniela florescu
2015-05-20 20:20:49 UTC
Permalink
Ok this is anecdotal and on a somewhat different tack but a couple of months ago I took the Marklogic Developer course. One of the few things I retained from the course was the realisation that the instructor had no idea what was in the XQuery specifiation (1.0 or 3.0) and in what way they differed from the Marklogic variant.
Ihe,

what I find not only pathetic, but totally reprehensible, in MarkLogic’s way of conducting business is:

(1) they don’t tell their customers that all the technology they use is based on an industrial standard in which
about 15 (years) times 20-30 (people = 3000-4500 man/year were invested by various companies : XQuery.

The advantages (flexibility, integration, blah, blah) that MarkLogic sells to its customers come mostly from the design of XQuery, not from anywhere else.

Yet, most customers never heard of XQuery. I talked to many of them. Users of XQuery for XXX years, yet they never heard of XQuery from
MarkLogic
.

That’s very reprehensible business behavior: it’s called lack of scientific and intellectual irresponsibility. And it’s done ON PURPOSE.

( Even worse, I think even their own salesforce has no clue about XQuery
..)


(2) just look at the www.marklogic.com <http://www.marklogic.com/>

The first who spots the word “XQuery” somewhere, gets a special cookie from me :-)

I wasn’t able to find it
.

How would the users be able to figure that MarkLogic didn’t invent ANY that !? You take them wining, dining and golfing, and 
 here you go.

(3) look at MarkLogic’s CEO, Gary Bloom, explaining to it’s customers about the “beauty” of MarkLogic:

http://youtu.be/EgBDSzak6jQ

Gary, no kidding man, you realized (finally!) that XQuery is used for data heterogeneous, schema-less data processing, for data integration and such
..!!!!!

WOW. I am impressed.

You finally got what the main design goal of XQuery WAS 
.. It’s only 20 years TOO late :-)

Sorry, Gary, the fact that you spent your life in Oracle doesn’t excuse your ignorance in managing heterogeneous data.

====================

Gary Bloom, when you say “WE have a solution” 
. you mean ****** “XQuery community has a solution”**********


Bring up some intellectual honesty, Gary, and stop bullshitting your customers.

Dana
daniela florescu
2015-05-09 18:31:45 UTC
Permalink
Michael,

I think I didn’t make myself clear.

The purpose of a standard is to increase interoperability in the world.

The decisions that were taken in the case of XQuery 3.1 do exactly the opposite, WITHOUT bringing in any major technical
advantages.

That’s all I am trying to say.

JSONiq is 3 years old, implemented by a variety of systems, and used in production in many places. And it was very well
designed: 2 years of VERY CAREFUL design by very good engineers went into the design of JSONiq.

Plus JSONiq will CONTINUE to be used widely in the NOSQL community, simply for the following two reasons:
(a) it has a JSON-only subset that makes sense for the JSON-only community (who in general doesn’t want to see any shadow of angle brackets) and
(b) has the great advantage that doesn’t have the hated word “Xquery” in its name.


By not following JSONiq, by taking “slightly different” choices, or by not trying to extend it, XQuery 3.1 just broke the interoperability
in the NOSQL community, and this is what I consider a total lack of vision from the “leaders” of this community.

Let’s look again of the differences you mentioned between JSONiq and XQuery 3.1. I quote:
***********************************************
At the data model level:

* Maps can use any atomic value as the key, it does not have to be a string
* The members of an array are sequences, not necessarily items
* JSON’s null is represented as an empty sequence, not as a new atomic data type

At the syntax level:

* In XQ3.1, Map constructors use the syntax map{ a:b, c:d } rather than bare curlies
* XQ 3.1 introduces a lookup operator for maps and arrays: employee?name, or book?author?1
**********************************************


Now, dear Michael, ask yourself the following questions:


*********************************************
1. Was there any “mistake”, or “bug”, that was in JSONiq that those choices solved ?
I am not aware of any.

2. Are any of those choices “fundamental’, aka introduce a much larger expressive power, make the language much easier to use, etc.?
Anything fundamental in those “new” choices ?
Nope, they are just small, mostly insignificant, just enough itsy bitsy differences to confuse and irritate the entire NoSQL community.

3. In case where a larger expressive power was desired, wasn’t it better to extend JSONiq in an upper compatible way, rather to take a different route ?

4. Did XQuery 3.1 “committee” ever tried to contact the JSONiq community to try to find common solution that would eventual lead to a single, better language ?
ALL decisions we took in the design of JSONiq were definitely not random, but VERY carefully thought out. You were not even curious of what those reason where,
(and sometimes XQuery 3.1 got it totally wrong — I an write you a long email about THAT).
*********************************************

So: no bugs to be fixed, no new major technical advantages, and no desire to cooperate.

What conclusion do you get from here ?

I get a single conclusion: the XQuery 3.1 committee has a total lack of vision and leadership in the NoSQL world.

This can has two possible explanations: pure stupidity and lack of good will.

I am trying to be nice, and pick the second choice among those two…..you see.

No wonder that because of silly decisions like this they will make XQuery to become irrelevant in the NoSQL world. No wonder the Cassandras, the Mongos
and the BigTable and all other JSON databases of the world will ignore what you “decided”, deal Michael Kay and other W3C people.


And that’s a pity because XQuery could have contributed quite a bit to the NoSQL world. XQuery has an experience of 16 years of processing semi-structured
data that those guys have no clue about (most they don’t even understand what they don’t know).

You made a HUGE political mistake in the design of XQuery 3.1. by fractioning the community.

And this: for no technical gain AT ALL.

The community looses, and I can see only two “people” who beneficiate (only short term) from this schism: Saxon and MarkLogic.

Long term everyone looses from those stupid decisions.

Dana
Post by Michael Kay
Post by daniela florescu
So, to me, the decisions of the W3C working group seems random, and rather based on a two years old
kind of a tantrum “I WANT TO BE DIFFERETENT JUST BECAUSE…..I WANT IT."
……...rather then justified by any technical reasons.
* generalizing maps to allow any atomic type as the key, rather than only a string, was because of specific use cases that required this (remember that the first proposal to add maps to XDM came from XSLT streaming work, not from JSONiq)
* the decision to use “map{…}” rather than “{…}” was to some extent subjective, but was motivated by technical arguments such as the ability to produce good error messages, retaining options for future extensions to the grammar, etc. Expressions beginning with “{“ are particularly problematic because “{“ is used to delimit embedded expressions in element content, and “{{“ is used to escape “{“ as an ordinary character; they are also very obscure when used as the body of a function (declare function f {{1:2}}). Before making this decision, we looked at how many other popular languages solve this problem.
* the decision to allow any sequence to act as a member of an array enabled things like the fn:apply() function, whose second argument is an array of arbitrary sequences; it also enabled JSON null to be represented by an empty sequence, which avoided the need for pervasive changes to the language to define how every function and operator should handle a JSON null. Reducing the number of concepts by one is a definite plus.
Getting agreement on all these points was a very lengthy process with much heated argument. Although the decisions made were not always the ones I personally advocated, I think the final language works well. If there’s one aspect I’m still a little unhappy about, it’s the fact that an array behaves like a single item, so for example
let $A := [1,2,3]
return $A[1]
returns [1,2,3]
But that’s there because we tried very hard to find a way to avoid this surprise, and failed: the sequence=item model in XDM is just too deeply embedded.
Michael Kay
Saxonica
_______________________________________________
***@x-query.com
http://x-query.com/mailman/
daniela florescu
2015-05-09 18:44:09 UTC
Permalink
BTW,

not only I don’t think that XQuery 3.1 didn’t bring any major technical advantage compared to JSONiq, I think some of the choices are simply
damn WRONG, and with long term negative consequences.

I can write a long email about that, but I don’t have time, and honestly, now I care more about Augmented Reality and 3D graphics
then I care about XML and JSON.

I will mention only the biggest mistake I see: the modeling of the JSON NULL value as an empty sequence.

Do you really think we were silly in the design of JSONiq when we made it a separate value, different from all other values !? No, we did it on purpose,
and with avery clear goal : to be able to control it’s semantics.

If you map the JSON NULL into the empty sequence, then the NULL will automatically behave in all operations (comparisons, arithmetics, etc)
as the empty sequence does in XQuery.

Unfortunately the semantics that XQuery has for the empty sequence is ***different*** from the semantics of NULL in the NATIVE language of JSON: Javascript.

Right there, by this “small” decision, you made XQuery 3.1 unusable, hated, and unused in the Javascript community, and you made the XML
community to NOT be able to work together with the JSON community.


Great achievement, lots of thinking…..W3C people ….. (it’s ironic..)

Dana
Post by daniela florescu
Michael,
I think I didn’t make myself clear.
The purpose of a standard is to increase interoperability in the world.
The decisions that were taken in the case of XQuery 3.1 do exactly the opposite, WITHOUT bringing in any major technical
advantages.
That’s all I am trying to say.
JSONiq is 3 years old, implemented by a variety of systems, and used in production in many places. And it was very well
designed: 2 years of VERY CAREFUL design by very good engineers went into the design of JSONiq.
(a) it has a JSON-only subset that makes sense for the JSON-only community (who in general doesn’t want to see any shadow of angle brackets) and
(b) has the great advantage that doesn’t have the hated word “Xquery” in its name.
By not following JSONiq, by taking “slightly different” choices, or by not trying to extend it, XQuery 3.1 just broke the interoperability
in the NOSQL community, and this is what I consider a total lack of vision from the “leaders” of this community.
***********************************************
* Maps can use any atomic value as the key, it does not have to be a string
* The members of an array are sequences, not necessarily items
* JSON’s null is represented as an empty sequence, not as a new atomic data type
* In XQ3.1, Map constructors use the syntax map{ a:b, c:d } rather than bare curlies
* XQ 3.1 introduces a lookup operator for maps and arrays: employee?name, or book?author?1
**********************************************
*********************************************
1. Was there any “mistake”, or “bug”, that was in JSONiq that those choices solved ?
I am not aware of any.
2. Are any of those choices “fundamental’, aka introduce a much larger expressive power, make the language much easier to use, etc.?
Anything fundamental in those “new” choices ?
Nope, they are just small, mostly insignificant, just enough itsy bitsy differences to confuse and irritate the entire NoSQL community.
3. In case where a larger expressive power was desired, wasn’t it better to extend JSONiq in an upper compatible way, rather to take a different route ?
4. Did XQuery 3.1 “committee” ever tried to contact the JSONiq community to try to find common solution that would eventual lead to a single, better language ?
ALL decisions we took in the design of JSONiq were definitely not random, but VERY carefully thought out. You were not even curious of what those reason where,
(and sometimes XQuery 3.1 got it totally wrong — I an write you a long email about THAT).
*********************************************
So: no bugs to be fixed, no new major technical advantages, and no desire to cooperate.
What conclusion do you get from here ?
I get a single conclusion: the XQuery 3.1 committee has a total lack of vision and leadership in the NoSQL world.
This can has two possible explanations: pure stupidity and lack of good will.
I am trying to be nice, and pick the second choice among those two…..you see.
No wonder that because of silly decisions like this they will make XQuery to become irrelevant in the NoSQL world. No wonder the Cassandras, the Mongos
and the BigTable and all other JSON databases of the world will ignore what you “decided”, deal Michael Kay and other W3C people.
And that’s a pity because XQuery could have contributed quite a bit to the NoSQL world. XQuery has an experience of 16 years of processing semi-structured
data that those guys have no clue about (most they don’t even understand what they don’t know).
You made a HUGE political mistake in the design of XQuery 3.1. by fractioning the community.
And this: for no technical gain AT ALL.
The community looses, and I can see only two “people” who beneficiate (only short term) from this schism: Saxon and MarkLogic.
Long term everyone looses from those stupid decisions.
Dana
Post by Michael Kay
Post by daniela florescu
So, to me, the decisions of the W3C working group seems random, and rather based on a two years old
kind of a tantrum “I WANT TO BE DIFFERETENT JUST BECAUSE…..I WANT IT."
……...rather then justified by any technical reasons.
* generalizing maps to allow any atomic type as the key, rather than only a string, was because of specific use cases that required this (remember that the first proposal to add maps to XDM came from XSLT streaming work, not from JSONiq)
* the decision to use “map{…}” rather than “{…}” was to some extent subjective, but was motivated by technical arguments such as the ability to produce good error messages, retaining options for future extensions to the grammar, etc. Expressions beginning with “{“ are particularly problematic because “{“ is used to delimit embedded expressions in element content, and “{{“ is used to escape “{“ as an ordinary character; they are also very obscure when used as the body of a function (declare function f {{1:2}}). Before making this decision, we looked at how many other popular languages solve this problem.
* the decision to allow any sequence to act as a member of an array enabled things like the fn:apply() function, whose second argument is an array of arbitrary sequences; it also enabled JSON null to be represented by an empty sequence, which avoided the need for pervasive changes to the language to define how every function and operator should handle a JSON null. Reducing the number of concepts by one is a definite plus.
Getting agreement on all these points was a very lengthy process with much heated argument. Although the decisions made were not always the ones I personally advocated, I think the final language works well. If there’s one aspect I’m still a little unhappy about, it’s the fact that an array behaves like a single item, so for example
let $A := [1,2,3]
return $A[1]
returns [1,2,3]
But that’s there because we tried very hard to find a way to avoid this surprise, and failed: the sequence=item model in XDM is just too deeply embedded.
Michael Kay
Saxonica
_______________________________________________
***@x-qu
Benito van der Zander
2015-05-09 20:52:27 UTC
Permalink
Hi,
so many recipients. Tempting to add even more.

Too bad it was not discussed on these lists when XQuery 3.1 was still a
draft.


I think the array constructor was the biggest mistake. (I believe
arrays more frequently used than null)
I doubt it would be possible to make a worse definition.

Three constructors, map { ... }, array { ... }, [ ... ]

Only [ ... ] looks out of place.

Every other constructor in XQuery uses { }. Even node constructors.

There is no reason to add [ .. ] for arrays, if you want to make a
consistent language.
Especially adding [ ... ] but then not a raw { ... } for objects.


Anyways, assume someone really wants to have an abbreviation, because
they like JSON or JSONiq.

How would you define the semantics of the constructors?

Use comma to separate members or as XQuery comma operator concatenating
sequences?


What does it do in JSON?

There are no sequences

What does it do in JSONiq's [ ... ]?

It does concatenate sequences.

What does the other new constructor similar to array { } do ? map { 1 :
(2,3), 4: (5,6) }

There , does not concatenate sequences.


So the only reasonable definition is to concat sequences in [ ] and not
concatenate them in array { }


But no, you do the complete opposite!



Conclusion: The [ .. ] constructor was just put there to annoy JSONiq
users. That is the only explanation.

So an usual query that creates arrays cannot be a valid JSONiq and
XQuery 3.1 query at the same time.



Best,
Benito
Post by daniela florescu
BTW,
not only I don’t think that XQuery 3.1 didn’t bring any major technical advantage compared to JSONiq, I think some of the choices are simply
damn WRONG, and with long term negative consequences.
I can write a long email about that, but I don’t have time, and honestly, now I care more about Augmented Reality and 3D graphics
then I care about XML and JSON.
I will mention only the biggest mistake I see: the modeling of the JSON NULL value as an empty sequence.
Do you really think we were silly in the design of JSONiq when we made it a separate value, different from all other values !? No, we did it on purpose,
and with avery clear goal : to be able to control it’s semantics.
If you map the JSON NULL into the empty sequence, then the NULL will automatically behave in all operations (comparisons, arithmetics, etc)
as the empty sequence does in XQuery.
Unfortunately the semantics that XQuery has for the empty sequence is ***different*** from the semantics of NULL in the NATIVE language of JSON: Javascript.
Right there, by this “small” decision, you made XQuery 3.1 unusable, hated, and unused in the Javascript community, and you made the XML
community to NOT be able to work together with the JSON community.
Great achievement, lots of thinking…..W3C people ….. (it’s ironic..)
Dana
Post by daniela florescu
Michael,
I think I didn’t make myself clear.
The purpose of a standard is to increase interoperability in the world.
The decisions that were taken in the case of XQuery 3.1 do exactly the opposite, WITHOUT bringing in any major technical
advantages.
That’s all I am trying to say.
JSONiq is 3 years old, implemented by a variety of systems, and used in production in many places. And it was very well
designed: 2 years of VERY CAREFUL design by very good engineers went into the design of JSONiq.
(a) it has a JSON-only subset that makes sense for the JSON-only community (who in general doesn’t want to see any shadow of angle brackets) and
(b) has the great advantage that doesn’t have the hated word “Xquery” in its name.
By not following JSONiq, by taking “slightly different” choices, or by not trying to extend it, XQuery 3.1 just broke the interoperability
in the NOSQL community, and this is what I consider a total lack of vision from the “leaders” of this community.
***********************************************
* Maps can use any atomic value as the key, it does not have to be a string
* The members of an array are sequences, not necessarily items
* JSON’s null is represented as an empty sequence, not as a new atomic data type
* In XQ3.1, Map constructors use the syntax map{ a:b, c:d } rather than bare curlies
* XQ 3.1 introduces a lookup operator for maps and arrays: employee?name, or book?author?1
**********************************************
*********************************************
1. Was there any “mistake”, or “bug”, that was in JSONiq that those choices solved ?
I am not aware of any.
2. Are any of those choices “fundamental’, aka introduce a much larger expressive power, make the language much easier to use, etc.?
Anything fundamental in those “new” choices ?
Nope, they are just small, mostly insignificant, just enough itsy bitsy differences to confuse and irritate the entire NoSQL community.
3. In case where a larger expressive power was desired, wasn’t it better to extend JSONiq in an upper compatible way, rather to take a different route ?
4. Did XQuery 3.1 “committee” ever tried to contact the JSONiq community to try to find common solution that would eventual lead to a single, better language ?
ALL decisions we took in the design of JSONiq were definitely not random, but VERY carefully thought out. You were not even curious of what those reason where,
(and sometimes XQuery 3.1 got it totally wrong — I an write you a long email about THAT).
*********************************************
So: no bugs to be fixed, no new major technical advantages, and no desire to cooperate.
What conclusion do you get from here ?
I get a single conclusion: the XQuery 3.1 committee has a total lack of vision and leadership in the NoSQL world.
This can has two possible explanations: pure stupidity and lack of good will.
I am trying to be nice, and pick the second choice among those two…..you see.
No wonder that because of silly decisions like this they will make XQuery to become irrelevant in the NoSQL world. No wonder the Cassandras, the Mongos
and the BigTable and all other JSON databases of the world will ignore what you “decided”, deal Michael Kay and other W3C people.
And that’s a pity because XQuery could have contributed quite a bit to the NoSQL world. XQuery has an experience of 16 years of processing semi-structured
data that those guys have no clue about (most they don’t even understand what they don’t know).
You made a HUGE political mistake in the design of XQuery 3.1. by fractioning the community.
And this: for no technical gain AT ALL.
The community looses, and I can see only two “people” who beneficiate (only short term) from this schism: Saxon and MarkLogic.
Long term everyone looses from those stupid decisions.
Dana
Post by Michael Kay
Post by daniela florescu
So, to me, the decisions of the W3C working group seems random, and rather based on a two years old
kind of a tantrum “I WANT TO BE DIFFERETENT JUST BECAUSE…..I WANT IT."
……...rather then justified by any technical reasons.
* generalizing maps to allow any atomic type as the key, rather than only a string, was because of specific use cases that required this (remember that the first proposal to add maps to XDM came from XSLT streaming work, not from JSONiq)
* the decision to use “map{…}” rather than “{…}” was to some extent subjective, but was motivated by technical arguments such as the ability to produce good error messages, retaining options for future extensions to the grammar, etc. Expressions beginning with “{“ are particularly problematic because “{“ is used to delimit embedded expressions in element content, and “{{“ is used to escape “{“ as an ordinary character; they are also very obscure when used as the body of a function (declare function f {{1:2}}). Before making this decision, we looked at how many other popular languages solve this problem.
* the decision to allow any sequence to act as a member of an array enabled things like the fn:apply() function, whose second argument is an array of arbitrary sequences; it also enabled JSON null to be represented by an empty sequence, which avoided the need for pervasive changes to the language to define how every function and operator should handle a JSON null. Reducing the number of concepts by one is a definite plus.
Getting agreement on all these points was a very lengthy process with much heated argument. Although the decisions made were not always the ones I personally advocated, I think the final language works well. If there’s one aspect I’m still a little unhappy about, it’s the fact that an array behaves like a single item, so for example
let $A := [1,2,3]
return $A[1]
returns [1,2,3]
But that’s there because we tried very hard to find a way to avoid this surprise, and failed: the sequence=item model in XDM is just too deeply embedded.
Michael Kay
Saxonica
_______________________________________________
http://x-query.com/mailman/listinfo/talk
_______________________________________________
***@x-query.com
http://x-query.com/ma
Michael Kay
2015-05-09 21:22:40 UTC
Permalink
I think the array constructor was the biggest mistake. (I believe arrays more frequently used than null)
I doubt it would be possible to make a worse definition.
Three constructors, map { ... }, array { ... }, [ ... ]
Only [ ... ] looks out of place.
Every other constructor in XQuery uses { }. Even node constructors.
There is no reason to add [ .. ] for arrays, if you want to make a consistent language.
I agree with you, I’m not happy with this aspect of the design. But the decision was reached after lengthy arguments at a F2F meeting which I didn’t attend, so I couldn’t complain.

The problem is that neither construct does the whole job. [a,b,c] can only be used to create an array whose size is statically known, whereas array{X,Y,Z} can only be used to create an array whose members are single items.

But the underlying problem is really that arrays are second-class citizens in the language. FLWOR expressions, filter expressions, even the comma operator, work on sequences but not on arrays. We tried many different design approaches for arrays in attempting to tackle this problem, but this was the best we could do.
So the only reasonable definition is to concat sequences in [ ] and not concatenate them in array { }
But no, you do the complete opposite!
Yes. I tried to get it changed to be the other way around, and failed. Just in case Daniela thinks I got everything my way.

Michael Kay
Saxonica
_______________________________________________
***@x-query.com
htt
Michael Kay
2015-05-09 20:57:57 UTC
Permalink
Post by daniela florescu
I will mention only the biggest mistake I see: the modeling of the JSON NULL value as an empty sequence.
Do you really think we were silly in the design of JSONiq when we made it a separate value, different from all other values !? No, we did it on purpose,
and with avery clear goal : to be able to control it’s semantics.
If you map the JSON NULL into the empty sequence, then the NULL will automatically behave in all operations (comparisons, arithmetics, etc)
as the empty sequence does in XQuery.
You can be assured that we had lively debates on this topic, and there were good arguments to be made both ways: none of them at all silly.

The use of empty sequence to represent absence of a value is very pervasive across all the functions and operators of the language. Many people felt that having two different ways to represent the absence of a value would just be too bewildering and too complex, and that few users would understand the subtle differences between them. If null can be supplied as an operand to comparisons, arithmetics, and to functions, then we not only need to define what every function and operator does when null is passed, we probably need to extend the type system so a function signature can declare that an argument is “nullable xs:integer” (i.e. the union of xs:integer and null), and we then have to teach users the difference between xs:integer? and “nullable xs:integer”. You even have to consider the type null? which is either null or empty. And we would almost inevitably have some functions returning () and others returning null to mean essentially the same thing - no result. People would have to check the spec every time they can’t remember which variety of null a particular function uses.

We were aware, also, that XQuery has two equality operators, and Javascript also has two, and none of them quite do the same thing. We didn’t want four equality operators. Unambiguous mapping of the JSON data model to XDM was an objective, but replication of the semantics of Javascript operators wasn’t.

Michael Kay
Saxonica




_______________________________________________
***@x-
daniela florescu
2015-05-09 21:25:04 UTC
Permalink
Post by Michael Kay
You can be assured that we had lively debates on this topic,
After “living” in the XQuery W3C working group for 15 years….. I don’t doubt you had. I am just happy I was not there.

But, sorry, W3C took the wrong decision: in one instant with this decision, you lost the JSON community.
Post by Michael Kay
and there were good arguments to be made both ways: none of them at all silly.
We were aware, also, that XQuery has two equality operators, and Javascript also has two, and none of them quite do the same thing. We didn’t want four equality operators.
Do you really think that XQuery has only TWO equality operators ?

No. Only two are in the external syntax. Every user I know of XQuery adds more as functions, depending to his/her own application needs.

So, two, four, six, ten, what’s the difference ? (it’s like that saying “the wife had only two lovers..it’s not that bad..four would have been really bad...” )

When you see that it is more then two, three, then you make it parametrable.

Making null a separate value would have enabled any programming language which is a consumer of JSON (and they ALL are..) to add their OWN semantics,
according to their wishes.

It’s called flexibility, and making things parametrable, (especially for something as important as equality for semi-structured data, whose semantics is in the eye of the beholder),
instead of rigid, monolithic, authocratic design, which invariably makes 90% users unhappy.


But that’s only one of the things that I think XQuery 3.1 got wrong. There are others.


The major OTHER problem is that it hadn’t been designed (as JSONiq did..) to be a syntactic superset of XQuery 3.0 , but which can be subsetted NICELY into a
JSON-only language that would have been syntactically and semantically pleasant to a JSON-only community.


For JSONiq this was one of the major, major design goals.


Too bad, because such W3C decisions have implications on splitting the XML and JSON communities, and THAT, has implications on decades of technology……and billions
of dollars lost in industry by unnecessary frictions and fragmentation of the market.


Dana



_______________________________________________
***@x-query
Ihe Onwuka
2015-05-13 08:35:51 UTC
Permalink
Post by Michael Kay
You can be assured that we had lively debates on this topic,
After “living” in the XQuery W3C working group for 15 years
.. I don’t
doubt you had. I am just happy I was not there.
But, sorry, W3C took the wrong decision: in one instant with this
decision, you lost the JSON community.
Is this not a variant of the Worse is Better argument (or did I not
properly understand that essay...... probably).

The issue with many JSON people is that they don't seem to acknowledge the
need for interoperability with XML so the utility of a "bilingual language"
probably doesn't resonate.

Without wishing to appear informed can I ask a question.

Was the Crockfordesque faction represented in the composition of the WG or
was it (of necessity) a bunch of XML people talking about how to support
JSON?
daniela florescu
2015-05-13 08:55:43 UTC
Permalink
The issue with many JSON people is that they don't seem to acknowledge the need for interoperability with XML so the utility of a "bilingual language" probably doesn't resonate.
Ihe,

this is where the JSONiq authors (me included) we did spent a HUGE amount of energy for design: in the following compromise:

(1) try to find a superset of XQuery 3.0 (so still stay in the XML world), which can handle BOTH XML and JSON.
This resulting language was not always pretty to look at, as you can imagine, because it inherited the syntactic
quirks of BOTH XML and JSON, yet it is very powerful and useful, as you can imagine, if in an application you need to mix and match both.

but in the same time, this billingual (expressive-power rich, but not pretty to look at, and not simple) language:

(b) was able to be syntactically subsetted to a VERY SIMPLE language for querying/processing JSON-only. This subset was maintaining the “good” parts of XQuery
(composability, declarativity, functional nature, optimizability, expressive power), while being aesthetically pleasing to a JSON-only crowd .


So I don’t agree with you. With JSONiq, we did prove that it IS possible to serve BOTh community with one underlying “virtual machine”, if you want, and several syntaxes,
according to the “beauty is in the eye of the beholder” principle.
Without wishing to appear informed can I ask a question.
Was the Crockfordesque faction represented in the composition of the WG or was it (of necessity) a bunch of XML people talking about how to support JSON?
I wasn’t there, so I cannot tell for sure.

But, after spending 16 years in the W3C, I guess that , as usual, the crowd was dominated by XML people who pay little attention to where the world goes,
outside their little XML narrow interest.


That’s what I witnessed during my 16 years tenure within W3C XQuery Working Group.

And it wasn’t either productive, not funny.


Best regards
Dana




_______________________________________________
***@x-query.com
ht
Ihe Onwuka
2015-05-13 09:11:40 UTC
Permalink
---------- Forwarded message ----------
From: Ihe Onwuka <***@gmail.com>
Date: Wed, May 13, 2015 at 5:11 AM
Subject: Re: [xquery-talk] [xml-dev] Mistakes made in the design of XQuery
3.1
Post by Ihe Onwuka
The issue with many JSON people is that they don't seem to acknowledge
the need for interoperability with XML so the utility of a "bilingual
language" probably doesn't resonate.
Ihe,
this is where the JSONiq authors (me included) we did spent a HUGE amount
(1) try to find a superset of XQuery 3.0 (so still stay in the XML
world), which can handle BOTH XML and JSON.
This resulting language was not always pretty to look at, as you can
imagine, because it inherited the syntactic
quirks of BOTH XML and JSON, yet it is very powerful and useful, as you
can imagine, if in an application you need to mix and match both.
but in the same time, this billingual (expressive-power rich, but not
(b) was able to be syntactically subsetted to a VERY SIMPLE language for
querying/processing JSON-only. This subset was maintaining the “good” parts
of XQuery
(composability, declarativity, functional nature, optimizability,
expressive power), while being aesthetically pleasing to a JSON-only crowd .
Sounds like Worse is Better.
So I don’t agree with you. With JSONiq, we did prove that it IS possible
to serve BOTh community with one underlying “virtual machine”, if you want,
and several syntaxes,
according to the “beauty is in the eye of the beholder” principle.
You misread me. Though not an expert but I have written some JSONiq so I
can appreciate what you are saying.

When i say JSON people I mean people like Marc (from the linkedin
discussion you started) and some others I have worked with who believe all
the myths and think everything can and should be done in JSON. OK maybe
not that far out because such a person would never come to an XQuery WG but
you know the sort of people I mean.
daniela florescu
2015-05-13 09:31:56 UTC
Permalink
You misread me. Though not an expert but I have written some JSONiq so I can appreciate what you are saying.
When i say JSON people I mean people like Marc (from the linkedin discussion you started) and some others I have worked with who believe all the myths and think everything can and should be done in JSON. OK maybe not that far out because such a person would never come to an XQuery WG but you know the sort of people I mean.
Yes, there are plenty of people in the world who consider XML and everything related to it (including XQuery) evil…..and JSON is all they want to see.


There are PLENTY of those.

As you say, in order to “educate” those guys about how to avoid the traps of building a silly query language of their own for JSON/semi-structured data
(because querying semi-strcuturded data is NOT trivial and COMPLETELY different from querying flat, regular, relational data, so, no, SQL is NOT an answer…),
you need to be a little … “diplomat”… and understand THEIR needs, and talk THEIR language.

Stuff that the esteemed XQuery 3.1 WG had no clue how to do. I guess they didn’t care either.

Otherwise you end up with those hacks that are all the JSON query languages designed by Cassandra, MongoDb, etc, etc.

No expressive power, and, worse, no clean semantics. (“let’s run it, and see what this query means”…... kind of semantics..)

=============

Hence the subset of JSONiq, which is based on the rich expressive power of XQuery and has the clean semantics of XQuery, yet syntactically
had absolutely NO mentioning of angle brackets AT ALL.

Just pure, pure JSON.

We did a mistake though, and forgot to rename “for" into “from” and “return” into “select” though …


I am shocked when I hear people telling me that they don’t understand the semantics of

for $x in collection(“blah”)
return $x.a

but it is obviously clear to them the semantics of the following expression

from $x in collection(“blah”)
select $x.a


===========


What can I say !? We learned that habits are extremely strong for some people, and keywords hold a powerful meaning….


Best regards
Dana
_______________________________________________
***@x-query.com
http://x-query.com/mailman/listinfo/tal
Benito van der Zander
2015-05-13 12:05:40 UTC
Permalink
Hi,
Post by daniela florescu
As you say, in order to “educate” those guys about how to avoid the traps of building a silly query language of their own for JSON/semi-structured data
(because querying semi-strcuturded data is NOT trivial and COMPLETELY different from querying flat, regular, relational data, so, no, SQL is NOT an answer…),
you need to be a little … “diplomat”… and understand THEIR needs, and talk THEIR language.
Still JSONiq screwed up, when they did not think of an equivalent for
$element/(child1, child2)/foo

At least $object . ("child1", "child2") . foo should have been allowed
Post by daniela florescu
We did a mistake though, and forgot to rename “for" into “from” and “return” into “select” though …
And the XQuery WG made a mistake, when they did not added macros.

#define from for
#define select return

I like C


Best,
Benito
Post by daniela florescu
You misread me. Though not an expert but I have written some JSONiq so I can appreciate what you are saying.
When i say JSON people I mean people like Marc (from the linkedin discussion you started) and some others I have worked with who believe all the myths and think everything can and should be done in JSON. OK maybe not that far out because such a person would never come to an XQuery WG but you know the sort of people I mean.
Yes, there are plenty of people in the world who consider XML and everything related to it (including XQuery) evil…..and JSON is all they want to see.
There are PLENTY of those.
As you say, in order to “educate” those guys about how to avoid the traps of building a silly query language of their own for JSON/semi-structured data
(because querying semi-strcuturded data is NOT trivial and COMPLETELY different from querying flat, regular, relational data, so, no, SQL is NOT an answer…),
you need to be a little … “diplomat”… and understand THEIR needs, and talk THEIR language.
Stuff that the esteemed XQuery 3.1 WG had no clue how to do. I guess they didn’t care either.
Otherwise you end up with those hacks that are all the JSON query languages designed by Cassandra, MongoDb, etc, etc.
No expressive power, and, worse, no clean semantics. (“let’s run it, and see what this query means”…... kind of semantics..)
=============
Hence the subset of JSONiq, which is based on the rich expressive power of XQuery and has the clean semantics of XQuery, yet syntactically
had absolutely NO mentioning of angle brackets AT ALL.
Just pure, pure JSON.
We did a mistake though, and forgot to rename “for" into “from” and “return” into “select” though …
I am shocked when I hear people telling me that they don’t understand the semantics of
for $x in collection(“blah”)
return $x.a
but it is obviously clear to them the semantics of the following expression
from $x in collection(“blah”)
select $x.a
===========
What can I say !? We learned that habits are extremely strong for some people, and keywords hold a powerful meaning….
Best regards
Dana
_______________________________________________
http://x-query.com/mailman/listinfo/talk
_______________________________________________
***@x-query.com
http://x-query.com/mailman/listin
daniela florescu
2015-05-13 17:38:12 UTC
Permalink
Post by Benito van der Zander
And the XQuery WG made a mistake, when they did not added macros.
#define from for
#define select return
I like C
The unfortunate choice of those two keywords (“for” and “return” instead of “from” and “select’) in XQuery had
such a HUGE impact on adoption (or not) of XQuery, it’s amazing.

It’s not too late to add the macros…..and make XQuery understandable to people who only want to see

select… from ...

Best regards
Dana



_______________________________________________
***@x-query.com
http://x-query.com/mailma
Ihe Onwuka
2015-05-21 06:47:19 UTC
Permalink
The unfortunate choice of those two keywords (“for” and “return” instead
of “from” and “select’) in XQuery had
such a HUGE impact on adoption (or not) of XQuery, it’s amazing.
It’s not too late to add the macros
..and make XQuery understandable to
people who only want to see
select
 from ...
Big mistakes.

Not co-opting a hardcore JSON zealot. I mean look at what they have
achieved with propaganda and myths and then imagine what could be possible
if you could convert a couple of them. Then again it could be like trying
to switch someone's sexual orientation (it's worth pondering why that
comparison is apt).

Bigger mistake

Not calling it JQuery.

Nuff said.
daniela florescu
2015-05-23 01:17:27 UTC
Permalink
Post by Ihe Onwuka
Bigger mistake
Not calling it JQuery.\
Ihe,

the second mistake you mention, not calling it JQuery.

I wish we could
. but JQuery was used for something else, alas:(

We didn’t have that choice
..:(

But I love the name JSONiq. It’s nice, and it’s cool, and more original then JQuery.

If I remember correctly, it was Jonathan Robie’s idea !

Dana
Ihe Onwuka
2015-05-25 08:29:10 UTC
Permalink
No I meant not calling XQuery JQuery. I believe there was time for
that....... just imagine......
Post by Ihe Onwuka
Bigger mistake
Not calling it JQuery.\
Ihe,
the second mistake you mention, not calling it JQuery.
I wish we could
. but JQuery was used for something else, alas:(
We didn’t have that choice
..:(
But I love the name JSONiq. It’s nice, and it’s cool, and more original
then JQuery.
If I remember correctly, it was Jonathan Robie’s idea !
Dana
Ihe Onwuka
2015-05-28 09:01:48 UTC
Permalink
If XQuery had been called JQuery people would have rushed to learn it
before they realised what it was. By which time it would have been too late.
Post by Ihe Onwuka
No I meant not calling XQuery JQuery. I believe there was time for
that....... just imagine......
Post by Ihe Onwuka
Bigger mistake
Not calling it JQuery.\
Ihe,
the second mistake you mention, not calling it JQuery.
I wish we could
. but JQuery was used for something else, alas:(
We didn’t have that choice
..:(
But I love the name JSONiq. It’s nice, and it’s cool, and more original
then JQuery.
If I remember correctly, it was Jonathan Robie’s idea !
Dana
daniela florescu
2015-05-28 18:19:51 UTC
Permalink
Ihe,

I am not sure I understand what you mean


XQuery was baptized XQuery around 2001, when no JSON was around (yet)
.

True that JSON was created later in the same year 2001, however it did not become widely popular
about until much later.

The json.org <http://json.org/> WS was created in 2002.

We cold have called it TheQuery :-)

Dana
If XQuery had been called JQuery people would have rushed to learn it before they realised what it was. By which time it would have been too late.
No I meant not calling XQuery JQuery. I believe there was time for that....... just imagine......
Post by Ihe Onwuka
Bigger mistake
Not calling it JQuery.\
Ihe,
the second mistake you mention, not calling it JQuery.
I wish we could
. but JQuery was used for something else, alas:(
We didn’t have that choice
..:(
But I love the name JSONiq. It’s nice, and it’s cool, and more original then JQuery.
If I remember correctly, it was Jonathan Robie’s idea !
Dana
Ihe Onwuka
2015-05-28 18:59:23 UTC
Permalink
Post by daniela florescu
Ihe,
I am not sure I understand what you mean

Well I'm being slightly tongue in cheek wise after the event.
Post by daniela florescu
XQuery was baptized XQuery around 2001, when no JSON was around (yet)
.
No Java was ......
Post by daniela florescu
True that JSON was created later in the same year 2001, however it did
not become widely popular
about until much later.
....but Java was already very very popular.

Now supposing an individual had cobbled together a language in 10 days that
was so crap that it's leading protagonist actually had to write a book
highlighting it's good parts. Read the first paragraph here.

https://www.w3.org/community/webed/wiki/A_Short_History_of_JavaScript

Imagine if they left the name as LiveScript or ECMAScript and note in the
last sentence of paragraph 1 why they didn't.
daniela florescu
2015-05-28 20:17:04 UTC
Permalink
Post by daniela florescu
XQuery was baptized XQuery around 2001, when no JSON was around (yet)
.
No Java was 


Java had at that time it’s own query language, actually made by some people originally involved with XQuery.
https://docs.oracle.com/javaee/6/tutorial/doc/bnbtg.html
Post by daniela florescu
True that JSON was created later in the same year 2001, however it did not become widely popular
about until much later.
....but Java was already very very popular.
Now supposing an individual had cobbled together a language in 10 days that was so crap that it's leading protagonist actually had to write a book highlighting it's good parts. Read the first paragraph here.
https://www.w3.org/community/webed/wiki/A_Short_History_of_JavaScript <https://www.w3.org/community/webed/wiki/A_Short_History_of_JavaScript>
Imagine if they left the name as LiveScript or ECMAScript and note in the last sentence of paragraph 1 why they didn’t.
That’s funny and interesting from a historical point of view.

But that’s water under the bridge.

============

The question for me is what could XQuery — and all the hard years of heavy experience that we all acquired in querying and processing, indexing, etc for schema-less data during 18 years — bring to the world of NoSQL query languages, which, today, is pretty pathetic.


Best regards
Dana
Ihe Onwuka
2015-05-29 05:03:30 UTC
Permalink
But I see my role as an engineer to build good engineering solutions, not
to increase the IQ of the general population. That’s not my job.
But you are not producing solutions for other engineers.

If you tell a developer with zero engineering qualifications their design
is structurally flawed or that they are using the wrong tools they (or
their boss) will turn around and tell you that there are several ways of
doing things and theirs is equally valid...... and that's if they are being
polite.

Then a couple of epochs and a failed project later they will get feted for
writing a blog about how you shouldn't use what they used or shouldn't do
what they did. Most probably said blog will focus on the effects of what
they did because they probably still don't understand the cause.
daniela florescu
2015-05-29 05:26:05 UTC
Permalink
Ihe,

my response to your email did not make it though the list.... so your answer to my answer ..... is not understandable to anyone.

Dana
But I see my role as an engineer to build good engineering solutions, not to increase the IQ of the general population. That’s not my job.
But you are not producing solutions for other engineers.
If you tell a developer with zero engineering qualifications their design is structurally flawed or that they are using the wrong tools they (or their boss) will turn around and tell you that there are several ways of doing things and theirs is equally valid...... and that's if they are being polite.
Then a couple of epochs and a failed project later they will get feted for writing a blog about how you shouldn't use what they used or shouldn't do what they did. Most probably said blog will focus on the effects of what they did because they probably still don't understand the cause.
Rob Rucker
2015-05-29 16:51:42 UTC
Permalink
HI Daniela, just wanted to appreciate your efforts to describe the
various forces at work in the informatics field, and the persistent
storage area in particular. I teach information modeling at a
university and so deciding on technical directions and what to
emphasize is important to me. I hope other knowledgeable people respond
in kind, as I would like to hear from NOSQL protagonists as well.
cheers
rob rucker
Post by daniela florescu
Ihe,
my response to your email did not make it though the list.... so your
answer to my answer ..... is not understandable to anyone.
Dana
But I see my role as an engineer to build good engineering
solutions, not to increase the IQ of the general population.
That’s not my job.
But you are not producing solutions for other engineers.
If you tell a developer with zero engineering qualifications their
design is structurally flawed or that they are using the wrong tools
they (or their boss) will turn around and tell you that there are
several ways of doing things and theirs is equally valid...... and
that's if they are being polite.
Then a couple of epochs and a failed project later they will get
feted for writing a blog about how you shouldn't use what they used
or shouldn't do what they did. Most probably said blog will focus on
the effects of what they did because they probably still don't
understand the cause.
_______________________________________________
http://x-query.com/mailman/listinfo/talk
daniela florescu
2015-05-29 14:54:16 UTC
Permalink
But I see my role as an engineer to build good engineering solutions, not to increase the IQ of the general population. That’s not my job.
But you are not producing solutions for other engineers.
As often as I hear that statement, I ignore at equally often.

Stubborn as I am, yes, I will CONTINUE to produce good solutions, designed for ENGINEERS, not for morons without any
engineering qualifications (but who build complex database applications).

I’ll leave that task to MarkLogic to “satisfy" the moron population and offer them Javascript as a server side query language.

As usually those are in larger numbers then the good engineers, MarkLogic will make more money 
..I guess.

But that’s not my goal, so I’m fine with that.

Again, stubborn as I am, my goal is to design good solutions for good engineers.


Best regards
Dana
Ihe Onwuka
2015-05-30 06:40:08 UTC
Permalink
Post by daniela florescu
But that’s not my goal, so I’m fine with that.
Again, stubborn as I am, my goal is to design good solutions for good engineers.
Great. Well JSONiq is my tool of choice for dealing with JSON.

Can you provide any insights to the graph database landscape. A
conversation on xml-dev with Peter Hunsberger has persuaded me that a dual
representation (XML for serving data and graph for running algorithms) is
probably what I need but I prefer to have some basis for evaluation before
jumping in with any particular product.
daniela florescu
2015-05-30 16:42:25 UTC
Permalink
Post by Ihe Onwuka
Great. Well JSONiq is my tool of choice for dealing with JSON.
Can you provide any insights to the graph database landscape. A conversation on xml-dev with Peter Hunsberger has persuaded me that a dual representation (XML for serving data and graph for running algorithms) is probably what I need but I prefer to have some basis for evaluation before jumping in with any particular product.
Unfortunately I cannot help you here. I tried to keep up with the graph languages, but lost it at some
point. Things are moving too fast.

And unlike NoSQL query languages, where the situation is really pathetic, the graph languages and their implementations
are not that bad, I think. So there are reasonable choices...

Sorry for not helping,
Dana


P.S. 20 years ago I wrote a paper about a graph language (we thought at that time that semi-stuctured data will necessarily be a graph
)
which kind of influenced the design of SPARQL.
http://www.en.pms.ifi.lmu.de/publications/projektarbeiten/Felix.Weigel/xmlindex/material/fernandez97strudel.pdf <http://www.en.pms.ifi.lmu.de/publications/projektarbeiten/Felix.Weigel/xmlindex/material/fernandez97strudel.pdf>

But of course, this is no help to you, other then intellectual curiosity :-)
Ihe Onwuka
2015-05-30 17:45:04 UTC
Permalink
Post by Ihe Onwuka
Great. Well JSONiq is my tool of choice for dealing with JSON.
Can you provide any insights to the graph database landscape. A
conversation on xml-dev with Peter Hunsberger has persuaded me that a dual
representation (XML for serving data and graph for running algorithms) is
probably what I need but I prefer to have some basis for evaluation before
jumping in with any particular product.
Unfortunately I cannot help you here. I tried to keep up with the graph
languages, but lost it at some
point. Things are moving too fast.
And unlike NoSQL query languages, where the situation is really pathetic,
the graph languages and their implementations
are not that bad, I think. So there are reasonable choices...
Ok back to these NoSql offerings I have a question.

You are always telling these vendors that their products are not databases
they are data stores. Now from that and the limited amount i have read (it
is difficult to continue reading beyond the point where you know this is
not a product you want to use) it sounds like these products are little
more than glorified VSAM files.

I get why SQL is not an appropriate basis for a language for these data
stores but I don't yet see how we make the leap from XQuery being a
language for particular types of semi-structured data to it being the basis
of a query language suitable for semi-structured data in principle, unless
you are talking in very broad terms about the features such a query
language should have and what principles should guide it's design.
Post by Ihe Onwuka
P.S. 20 years ago I wrote a paper about a graph language (we thought at
that time that semi-stuctured data will necessarily be a graph
)
which kind of influenced the design of SPARQL.
http://www.en.pms.ifi.lmu.de/publications/projektarbeiten/Felix.Weigel/xmlindex/material/fernandez97strudel.pdf
But of course, this is no help to you, other then intellectual curiosity :-)
People who are not intellectually curious end up writing blogs about why
you should not do what they did. Then again they also get the kudos of
being invited to give talks about their cock-ups to other people who are
not intellectually curious but want to learn from the bitter experience of
others.

But yes i will try and give it a read.
daniela florescu
2015-05-30 18:12:23 UTC
Permalink
I will give you a somewhat short answer to that question but we can expand only if necessary.

**********
True in general for every query language
**********

1. In general I think a “query” language should be functional and compositional.
(besides having a high expressive power, it has other advantages like being able to do data flow analysis
through the function invocations — which is necessary for almost EVERYTHING inside a database,
from logging to index detection to optimizability)

SQL got only a “little” but of comparability only after decades of existence
.for my taste it’s really clunky.
I never remember what I can compose with what — and never understand why not !??

2. In general a query language should have the FLWOR-like expression (called it as you like, SELECT-FROM-WHERE
whatever). In mathematics (or theoretical computer science) it is called monoid comprehension.

The more powerful the FLWOR like expression, the more expressive power of the query language.

I think XQuery has the best designed expression of this kind in existence. It’s compositional, elegant, symmetrical.
The semantics is very well defined, and 
 if you look well
 NOWHERE IN THE DEFINITION OF THE SEMANTICS
OF THE FLOWR EXPRESSION THE WORD XML OCCURS.

The FLWOR expression of XQuery is a construct that has NOTHING to do with XML !!

3. In general, a query language is not a query language unless is optimizable.(aka the ability for a compiler to be able to
derive from one syntax many different ways of evaluating the result, some being more optimal then others).

I think during the design of XQuery a HUGE deal of attention was done to the optimizability of the language. The implementations
show that we didn’t get it wrong. Most implementations have decent response time (even though alas, there is no standard benchmark).


***************
Things that are absolutely necessary for semi-structured query languages
**************

1.Conditionals, typeswitches

In schema-less data, we never know what we’ll find. We HAVE to be able to branch based on what we find in the data while searching.

2. Functions and recursive functions

While dealing with nested, pre-aggregated data, we need to navigate structures on unknown depth, and for this we really need functions
and recursive functions.

3. Error management: try/catch

Even with the best precautions, while dealing with data of unknown structure, we can ALWAYS find unexpected stuff: an integer when we
thought we’ll get a string, and we try to cast it to a date. Hence, an error.

Imagine you processed already 1 million rows, and suddenly one of those rows is “bad”. In the absence of a try/catch the whole result will be an exception
and your hard work on a million rows is lost.


==========

Now those three things: switches of all kinds, recursive functions and error management change the good old query processing DRAMATICALLY.
(why is that ? Because they make difficult the famous data flow analysis I just talked about — and which is a necessity in any query optimizer).

Relational database optimizers don’t deal with such difficulties, because they didn’t have them.


That’s a short answer of why XQuery is better for processing semi-structured data then any other alternative out there.

Note: REMARK THAT IN THIS WHOLE EMAIL I NEVER PRONOUNCED THE WORD XML.

XQuery, despite it’s very unfortunate name, is NOT an XML query language (ONLY).

It’s basic principles have nothing to do with XML.

That’s why in JSONiq it was trivial to take out the “/“ navigation, replace it with “.” navigation, replace the node constructors with object
constructors, and here we go.


Best regards
Dana
Post by daniela florescu
Post by Ihe Onwuka
Great. Well JSONiq is my tool of choice for dealing with JSON.
Can you provide any insights to the graph database landscape. A conversation on xml-dev with Peter Hunsberger has persuaded me that a dual representation (XML for serving data and graph for running algorithms) is probably what I need but I prefer to have some basis for evaluation before jumping in with any particular product.
Unfortunately I cannot help you here. I tried to keep up with the graph languages, but lost it at some
point. Things are moving too fast.
And unlike NoSQL query languages, where the situation is really pathetic, the graph languages and their implementations
are not that bad, I think. So there are reasonable choices...
Ok back to these NoSql offerings I have a question.
You are always telling these vendors that their products are not databases they are data stores. Now from that and the limited amount i have read (it is difficult to continue reading beyond the point where you know this is not a product you want to use) it sounds like these products are little more than glorified VSAM files.
I get why SQL is not an appropriate basis for a language for these data stores but I don't yet see how we make the leap from XQuery being a language for particular types of semi-structured data to it being the basis of a query language suitable for semi-structured data in principle, unless you are talking in very broad terms about the features such a query language should have and what principles should guide it's design.
P.S. 20 years ago I wrote a paper about a graph language (we thought at that time that semi-stuctured data will necessarily be a graph
)
which kind of influenced the design of SPARQL.
http://www.en.pms.ifi.lmu.de/publications/projektarbeiten/Felix.Weigel/xmlindex/material/fernandez97strudel.pdf <http://www.en.pms.ifi.lmu.de/publications/projektarbeiten/Felix.Weigel/xmlindex/material/fernandez97strudel.pdf>
But of course, this is no help to you, other then intellectual curiosity :-)
People who are not intellectually curious end up writing blogs about why you should not do what they did. Then again they also get the kudos of being invited to give talks about their cock-ups to other people who are not intellectually curious but want to learn from the bitter experience of others.
But yes i will try and give it a read.
daniela florescu
2015-05-30 19:43:57 UTC
Permalink
Post by daniela florescu
***************
Things that are absolutely necessary for semi-structured query languages
**************
1.Conditionals, typeswitches
In schema-less data, we never know what we’ll find. We HAVE to be able to branch based on what we find in the data while searching.
2. Functions and recursive functions
While dealing with nested, pre-aggregated data, we need to navigate structures on unknown depth, and for this we really need functions
and recursive functions.
3. Error management: try/catch
In this list I forgot something really important for any kind of query language for semi-strcutured data;

4. A battery of implicit type conversions.

Data being of unexpected shape and type, we can never be sure that the data operations get at runtime is what you expected when you wrote the query.

Defining those is an **ART** (with capitals:-), because one need to make a compromise between:
- not being too strict, otherwise everything ends up in an error
- not being too lax ("anything goes" kind of thing), because you introduce errors in the resulting data
- making sure that the implicit type conversion are done such that you can STILL use indexes (if you index a value as an integer and at runtime it gets converted and
used as string in an equality… are you still able to use the index !?)
-usability and symmetry ; such implicit conversions happen in a variety of places (function calls, equality, arithmetics, groupby, sort, etc, etc) and you have to make sure that
they are kind of symmetrical and obey the “principle of least surprise” and also allow optimizability, aka expression rewriting

At least 25% of the design of XQuery has spent in the design of this battery of implicit type conversions.

I think the result is decent. I am not ware of any major mistake, that I wish we would have done better, or differently. But everything was design by trail, design, implementation, usage, error,
designe, implementation, etc, etc. That’s why it took an eternity to design them, and for XQuery to get standardized.

I honestly don’t wish even my worst enemy to start this process from scratch, and I wish the NoSQL community would have learned from what we’ve done.

Wishful thinking on Saturday morning, best regards
Dana




_______________________________________________
***@x-query.com
http://x-query.com/mailman/listinfo/tal
daniela florescu
2015-05-30 20:17:42 UTC
Permalink
Post by daniela florescu
I get why SQL is not an appropriate basis for a language for these data stores but I don't yet see how we make the leap from XQuery being a language for particular types of semi-structured data to it being the basis of a query language suitable for semi-structured data in principle, unless you are talking in very broad terms about the features such a query language should have and what principles should guide it's design.
Ihe,

I gave you a general answer in the previous emails.

Yet I have to remark something before I shut up.

Our work in JSONiq applies to JSON. Period.

Not all NoSQL databases use JSON as a data model (or syntax, whatever) - some use it just a simple
add-on, or import/export syntax, similar to CSV.

The problem of building a good query language for those NoSQL databases starts with the fact that
THEY HAVE NO CLEAR DATA MODEL of the data they store. Not even sometimes of what basic types
they can store (e.g. ask Cassandra people if they can store dates, and you’ll get a very convoluted answer…)

Most of the times the “data model” is just an implicit assumption behind the API calls that you can make to that database/datastore.

I don’t know how to build a good query language, unless I have a logical model of the data being queried… it’s that simple.

At least JSON… I understand JSON… so I can build a query language for it….

It’s not a value judgment (aka "JSON is better”), it’s just that it’s there, it’s well described, and people use it everywhere.

Best regards
Dana



_______________________________________________
***@x-query.com
http://x-query.com/mailman/listinf

daniela florescu
2015-05-29 05:28:43 UTC
Permalink
This email was sent this morning.

Obviously, for "different accounts" reasons, it didn't make it through. Yeah.

Dana
It proves that when it comes to IT you can put lipstick on a pig. If you were trying to name the language today JShit would probably market test better than XQuery.
That’s why I’m so tired of the database “science” and the database “scientific” community, and I run away to 3D and Augmented reality — which is at the beginning,
nobody makes billions (yet) and there is still some love of technology, some honesty and integrity, and deep, serious enthusiasm. There is still some innocence...
Right now the “database market” is just lipstick on a (VERY LARGE) pig. Lots of money spent on marketing scream, and unfortunately developers flock like sheep
towards the ones who scream the most, without any clue of “why”.
Yet we have no clue why a solution is better then other, no benchmarks, just hacky solutions, or temporary solutions which will disappear in 5-10 years, etc.
Maybe it's because of your abhorrence of stupidity you tend not to stick around long enough to witness just how stupid some people are.
Well, I am VERY patient when I want to.... I did stay with the XML community since 1997, despite my deep dislike for processing instructions... :-)
But I see my role as an engineer to build good engineering solutions, not to increase the IQ of the general population. That’s not my job.
Best regards
Dana
Ihe Onwuka
2015-05-29 05:29:09 UTC
Permalink
But you are not producing solutions for other engineers.

If you tell a developer with zero engineering qualifications their design
is structurally flawed or that they are using the wrong tools they (or
their boss) will turn around and tell you that there are several ways of
doing things and theirs is equally valid...... and that's if they are being
polite.

Then a couple of epochs and a failed project later they will get feted for
writing a blog about how you shouldn't use what they used or shouldn't do
what they did. Most probably said blog will focus on the effects of what
they did because they probably still don't understand the cause.
It proves that when it comes to IT you can put lipstick on a pig. If you
were trying to name the language today JShit would probably market test
better than XQuery.
That’s why I’m so tired of the database “science” and the database
“scientific” community, and I run away to 3D and Augmented reality — which
is at the beginning,
nobody makes billions (yet) and there is still some love of technology,
some honesty and integrity, and deep, serious enthusiasm. There is still
some innocence...
Right now the “database market” is just lipstick on a (VERY LARGE) pig.
Lots of money spent on marketing scream, and unfortunately developers flock
like sheep
towards the ones who scream the most, without any clue of “why”.
Yet we have no clue why a solution is better then other, no benchmarks,
just hacky solutions, or temporary solutions which will disappear in 5-10
years, etc.
Maybe it's because of your abhorrence of stupidity you tend not to stick
around long enough to witness just how stupid some people are.
Well, I am VERY patient when I want to.... I did stay with the XML
community since 1997, despite my deep dislike for processing
instructions... :-)
But I see my role as an engineer to build good engineering solutions, not
to increase the IQ of the general population. That’s not my job.
Best regards
Dana
daniela florescu
2015-05-29 05:49:43 UTC
Permalink
If you tell a developer with zero engineering qualifications their design is structurally flawed
Ihe,

sorry.....in this story **I** not the one to blame. And I am not Mother Theresa either.

A developer with ZERO engineering qualifications (as you say..) should DAMN SHUT UP, get a job as a carpenter..... AND NOT BUILD
SERIOUS DATABASE APPLICATIONS.

Writing code, and writing complex software requires engineering qualifications, sorry.


As one of my Linkedin friends a couple of days back, in a funny way: " MongoDB is a living proof that some people never go from
high school to university."

Again, I am not Mother Theresa, nor Jesus to sacrifice myself for the fact that most developers have no engineering qualifications.


Too bad for the companies who hire them. What can I say!???


Best regards
Dana
daniela florescu
2015-05-29 05:55:37 UTC
Permalink
How about starting tomorrow we allow random people from the street, without any medical qualifications, to start
doing heart surgery at the Stanford Medical School !???

Or we allow anybody from the street, without any architectural or structural mechanics qualifications to start building bridges in San Francisco !?

Or tomorrow I'll show up at the San Francisco airport and tell them that I'm the one piloting the plane to New York !?

===========

Bottom line...only in software such an aberration is tolerated.....

Dana
Post by Ihe Onwuka
But you are not producing solutions for other engineers.
If you tell a developer with zero engineering qualifications their design is structurally flawed or that they are using the wrong tools they (or their boss) will turn around and tell you that there are several ways of doing things and theirs is equally valid...... and that's if they are being polite.
Then a couple of epochs and a failed project later they will get feted for writing a blog about how you shouldn't use what they used or shouldn't do what they did. Most probably said blog will focus on the effects of what they did because they probably still don't understand the cause.
It proves that when it comes to IT you can put lipstick on a pig. If you were trying to name the language today JShit would probably market test better than XQuery.
That’s why I’m so tired of the database “science” and the database “scientific” community, and I run away to 3D and Augmented reality — which is at the beginning,
nobody makes billions (yet) and there is still some love of technology, some honesty and integrity, and deep, serious enthusiasm. There is still some innocence...
Right now the “database market” is just lipstick on a (VERY LARGE) pig. Lots of money spent on marketing scream, and unfortunately developers flock like sheep
towards the ones who scream the most, without any clue of “why”.
Yet we have no clue why a solution is better then other, no benchmarks, just hacky solutions, or temporary solutions which will disappear in 5-10 years, etc.
Maybe it's because of your abhorrence of stupidity you tend not to stick around long enough to witness just how stupid some people are.
Well, I am VERY patient when I want to.... I did stay with the XML community since 1997, despite my deep dislike for processing instructions... :-)
But I see my role as an engineer to build good engineering solutions, not to increase the IQ of the general population. That’s not my job.
Best regards
Dana
_______________________________________________
http://x-query.com/mailman/listinfo/talk
Ihe Onwuka
2015-05-29 08:34:16 UTC
Permalink
Post by daniela florescu
How about starting tomorrow we allow random people from the street,
without any medical qualifications, to start
doing heart surgery at the Stanford Medical School !???
Or we allow anybody from the street, without any architectural or
structural mechanics qualifications to start building bridges in San
Francisco !?
Or tomorrow I'll show up at the San Francisco airport and tell them that
I'm the one piloting the plane to New York !?
===========
Bottom line...only in software such an aberration is tolerated.....
Because unlike in the other instances the true effect of software designed
by amateurs lies latent and is usually neither immediately nor outwardly
apparent.

Tolerated is the wrong word. The entire industry has coalesced around
technologies and methodologies designed to enable the amateur coder to
write professional software and these are the people that determine the
fate of your engineering solution and what tools and methods you are going
to have to work with. Look at the recent changes in MarkLogic and you can
clearly see where they try to cater to that demographic. Or look at Apache
Spark - built in Scala - do you think their support for Scala emanates from
engineering considerations.

So then you have an environment in which it is more important to know how
to use these tools and methods than it is to have a fundamental
understanding of what it is you are doing. As an example (he says
deliberately avoiding ones close to home), look at Data Science. Industry
is such that it is more important to know how to program in R than it is to
have a basic understanding of Statistics, but the nature of the output they
produce is such that the effects of such practices will remain camouflaged
for a long time to come (or until the UK has another general election). R
in particular is one to watch, because of all the hype you have a
confluence of people who know little about Statistics and even less about
programming.

I could go on..... the reality is that today people today want to query
semi-structured data in Javascript, Python and R and you can't look to
their management because this is the first generation that grew up with
managers that did not have a computer science or engineering background.

So sadly Daniela, politics and populism trump engineering and not enough
lives are at risk in what we do for that to ever change. The choice is to
come to terms with it, go back to research or find something else to do.
Post by daniela florescu
Dana
But you are not producing solutions for other engineers.
If you tell a developer with zero engineering qualifications their design
is structurally flawed or that they are using the wrong tools they (or
their boss) will turn around and tell you that there are several ways of
doing things and theirs is equally valid...... and that's if they are being
polite.
Then a couple of epochs and a failed project later they will get feted for
writing a blog about how you shouldn't use what they used or shouldn't do
what they did. Most probably said blog will focus on the effects of what
they did because they probably still don't understand the cause.
It proves that when it comes to IT you can put lipstick on a pig. If you
were trying to name the language today JShit would probably market test
better than XQuery.
That’s why I’m so tired of the database “science” and the database
“scientific” community, and I run away to 3D and Augmented reality — which
is at the beginning,
nobody makes billions (yet) and there is still some love of technology,
some honesty and integrity, and deep, serious enthusiasm. There is still
some innocence...
Right now the “database market” is just lipstick on a (VERY LARGE) pig.
Lots of money spent on marketing scream, and unfortunately developers flock
like sheep
towards the ones who scream the most, without any clue of “why”.
Yet we have no clue why a solution is better then other, no benchmarks,
just hacky solutions, or temporary solutions which will disappear in 5-10
years, etc.
Maybe it's because of your abhorrence of stupidity you tend not to stick
around long enough to witness just how stupid some people are.
Well, I am VERY patient when I want to.... I did stay with the XML
community since 1997, despite my deep dislike for processing
instructions... :-)
But I see my role as an engineer to build good engineering solutions, not
to increase the IQ of the general population. That’s not my job.
Best regards
Dana
_______________________________________________
http://x-query.com/mailman/listinfo/talk
Ihe Onwuka
2015-05-29 08:35:55 UTC
Permalink
Post by Ihe Onwuka
Post by daniela florescu
How about starting tomorrow we allow random people from the street,
without any medical qualifications, to start
doing heart surgery at the Stanford Medical School !???
Or we allow anybody from the street, without any architectural or
structural mechanics qualifications to start building bridges in San
Francisco !?
Or tomorrow I'll show up at the San Francisco airport and tell them that
I'm the one piloting the plane to New York !?
===========
Bottom line...only in software such an aberration is tolerated.....
Because unlike in the other instances the true effect of software designed
by amateurs lies latent and is usually neither immediately nor outwardly
apparent.
Tolerated is the wrong word. The entire industry has coalesced around
technologies and methodologies designed to enable the amateur coder to
write professional software and these are the people that determine the
fate of your engineering solution and what tools and methods you are going
to have to work with. Look at the recent changes in MarkLogic and you can
clearly see where they try to cater to that demographic. Or look at Apache
Spark - built in Scala - do you think their support for Scala emanates from
engineering considerations.
that should read ... do you think their support for Python emanates from
engineering considerations.
Post by Ihe Onwuka
So then you have an environment in which it is more important to know how
to use these tools and methods than it is to have a fundamental
understanding of what it is you are doing. As an example (he says
deliberately avoiding ones close to home), look at Data Science. Industry
is such that it is more important to know how to program in R than it is to
have a basic understanding of Statistics, but the nature of the output they
produce is such that the effects of such practices will remain camouflaged
for a long time to come (or until the UK has another general election). R
in particular is one to watch, because of all the hype you have a
confluence of people who know little about Statistics and even less about
programming.
I could go on..... the reality is that today people today want to query
semi-structured data in Javascript, Python and R and you can't look to
their management because this is the first generation that grew up with
managers that did not have a computer science or engineering background.
So sadly Daniela, politics and populism trump engineering and not enough
lives are at risk in what we do for that to ever change. The choice is to
come to terms with it, go back to research or find something else to do.
Post by daniela florescu
Dana
But you are not producing solutions for other engineers.
If you tell a developer with zero engineering qualifications their design
is structurally flawed or that they are using the wrong tools they (or
their boss) will turn around and tell you that there are several ways of
doing things and theirs is equally valid...... and that's if they are being
polite.
Then a couple of epochs and a failed project later they will get feted
for writing a blog about how you shouldn't use what they used or shouldn't
do what they did. Most probably said blog will focus on the effects of what
they did because they probably still don't understand the cause.
It proves that when it comes to IT you can put lipstick on a pig. If you
were trying to name the language today JShit would probably market test
better than XQuery.
That’s why I’m so tired of the database “science” and the database
“scientific” community, and I run away to 3D and Augmented reality — which
is at the beginning,
nobody makes billions (yet) and there is still some love of technology,
some honesty and integrity, and deep, serious enthusiasm. There is still
some innocence...
Right now the “database market” is just lipstick on a (VERY LARGE) pig.
Lots of money spent on marketing scream, and unfortunately developers flock
like sheep
towards the ones who scream the most, without any clue of “why”.
Yet we have no clue why a solution is better then other, no benchmarks,
just hacky solutions, or temporary solutions which will disappear in 5-10
years, etc.
Maybe it's because of your abhorrence of stupidity you tend not to stick
around long enough to witness just how stupid some people are.
Well, I am VERY patient when I want to.... I did stay with the XML
community since 1997, despite my deep dislike for processing
instructions... :-)
But I see my role as an engineer to build good engineering solutions,
not to increase the IQ of the general population. That’s not my job.
Best regards
Dana
_______________________________________________
http://x-query.com/mailman/listinfo/talk
Ihe Onwuka
2015-05-29 08:51:27 UTC
Permalink
Post by Ihe Onwuka
Post by Ihe Onwuka
I could go on..... the reality is that today people today want to query
semi-structured data in Javascript, Python and R and you can't look to
their management because this is the first generation that grew up with
managers that did not have a computer science or engineering background.
Honourable mentions should go out to the hard sciences and quantitative
disciplines.
daniela florescu
2015-05-23 01:11:43 UTC
Permalink
The unfortunate choice of those two keywords (“for” and “return” instead of “from” and “select’) in XQuery had
such a HUGE impact on adoption (or not) of XQuery, it’s amazing.
It’s not too late to add the macros
..and make XQuery understandable to people who only want to see
select
 from ...
Big mistakes.
Ihe,

THAT particular choice of keywords (FOR/RETURN
 instead of FROM/SELECT)
. is MY mistake ENTIRELY. 100% MINE.

I regretted that (daily) in the past 16 years 
..and did LOTS of penance for it
..please believe me.

I could tell you the story
 maybe it’s fun, maybe it’s interesting from a technology/history perspective.


=====

It was 1999, and I was a researcher in INRIA, studying query languages and query processing for semi-structured data. (since 1996 when I was in ATT Research)
I was stuck in Paris, where nothing ever happened in technology, and I was dreaming of going to Sillicon Valley, where EVERYTHING happened.
So, I managed to become a visiting scientist at IBM Almaden in Sillicon Valley.
My boss at that time, Mike Carey (now professor it Univ. Irvine), just decided to leave IBM for a startup — Propel.
He didn’t know what to do with me (I was his responsibility :-) 
., so he took me by the hand to Don Chambelin (of SQL fame)
across the hall, and he asked me: “Why don’t you work with Don Chamberlin on this XML query thing, after I am gone ?”
That’s what I did, together with Jonathan Robie, and the result was Quilt, the precursor of XQuery.
Well, it happened that my PhD thesis was on query languages and query processing of object-oriented languages, and OQL was a functional language, above
and before being a query language. Hence, XQuery became a functional language.
Well, in the same time, I was teaching in a French university the ML programming language, a long love of mine, as programming languages are concerned.
Here is the catch: ML used the words FOR and RETURN for monoid comprehension
.which both the Select-From-Where of SQL and the FLOWR expression are.

Back to the story:

While in IBM, one evening Don Chamberlin asked me to write a grammar and a parser for Quilt. I spent the night doing that, and in the morning I showed him what I did.
As a fresh lover of ML, I used the keywords FOR and RETURN in the grammar.


Don looked at me and told me
. “This thing
.we’ll call it a FLWOR expression.”


=======

</end-of-story>

It NEVER crossed my mind RIGHT THEN that that random decision that I took one night in a state of extreme fatigue will become cut-in-stone within
the over-powerful standard that XQuery is today.


I guess lots of technical decisions happen this way
.randomly.


Best regards,
Dana
Ihe Onwuka
2015-05-25 09:09:29 UTC
Permalink
Post by Ihe Onwuka
The unfortunate choice of those two keywords (“for” and “return” instead
of “from” and “select’) in XQuery had
such a HUGE impact on adoption (or not) of XQuery, it’s amazing.
It’s not too late to add the macros
..and make XQuery understandable to
people who only want to see
select
 from ...
Big mistakes.
Ihe,
THAT particular choice of keywords (FOR/RETURN
 instead of FROM/SELECT)
.
is MY mistake ENTIRELY. 100% MINE.
I regretted that (daily) in the past 16 years 
..and did LOTS of penance
for it
..please believe me.
Methinks you are underestimating peoples hypocrisy. Take something like

Select album from music_Artist where name = "The Police"

This is what passes without complaint in MQL a JSON based query language
(example from http://wiki.freebase.com/wiki/MQL)

{
"type": "/music/artist",
"name": "The Police",
"album": []
}


If I may revert back to the SQL/NoSQL issue. People perceive SQL as just
that. A query language for querying structured data and in their minds
that subsumes semi structured data. They don't see it as a codification of
relational algebra that operates on relations. If they did they might be
more hesitant about expecting anything good to result from applying SQL to
things that are not relations. That however, would require the learning of
a little bit of theory and unfortunately we have to co-exist with people
who write code for a living but passionately believe that being uneducated
is a virtue. These same people propagate blogs illustrated with simplistic
examples and before you know it Stupid Is As Stupid Does becomes the
defacto paradigm.

So don't beat yourself up about it.
Post by Ihe Onwuka
I could tell you the story
 maybe it’s fun, maybe it’s interesting from a
technology/history perspective.
=====
It was 1999, and I was a researcher in INRIA, studying query languages and
query processing for semi-structured data. (since 1996 when I was in ATT
Research)
I was stuck in Paris, where nothing ever happened in technology, and I was
dreaming of going to Sillicon Valley, where EVERYTHING happened.
So, I managed to become a visiting scientist at IBM Almaden in Sillicon Valley.
My boss at that time, Mike Carey (now professor it Univ. Irvine), just
decided to leave IBM for a startup — Propel.
He didn’t know what to do with me (I was his responsibility :-) 
., so he
took me by the hand to Don Chambelin (of SQL fame)
across the hall, and he asked me: “Why don’t you work with Don
Chamberlin on this XML query thing, after I am gone ?”
That’s what I did, together with Jonathan Robie, and the result was Quilt,
the precursor of XQuery.
Well, it happened that my PhD thesis was on query languages and query
processing of object-oriented languages, and OQL was a functional language,
above
and before being a query language. Hence, XQuery became a functional language.
Well, in the same time, I was teaching in a French university the ML
programming language, a long love of mine, as programming languages are
concerned.
Here is the catch: ML used the words FOR and RETURN for monoid
comprehension
.which both the Select-From-Where of SQL and the FLOWR
expression are.
Is that also where we got typeswitch from... ML?
daniela florescu
2015-05-26 23:55:36 UTC
Permalink
Post by Ihe Onwuka
Is that also where we got typeswitch from... ML?
No, typeswitch was introduced to XQuery by Phil Wadler (of Haskell fame), and honestly, I was personally against it when I first saw it.

(I remember the scene like it was yesterday
 and Phil got really angry with me
and Adam Bosworth who was sitting next to me...)

We could thank Phil that he was persistent and ignored us 
 :-)


Dana
Ihe Onwuka
2015-05-16 16:59:10 UTC
Permalink
In passing, I would like to observe that the title of the W3C Working
Group being discussed is "XML Query Working Group" and the charter of that
WG speaks of developing a query language for XML. In fact, the language
developed by the WG is called "XQuery: An XML Query Language". While the
WG, and its participants, understand the importance of JSON and have as a
goal integration of support for querying XML data along with JSON data, the
name and charter of the WG have not been changed to "JSON Query WG".
Might that be because JSON (for all intents and purposes) did not exist at
the inception of the WG.

I mean it's not called the SQL Query WG either but the WG did have SQL as a
reference point because relational data formats were around and relevant at
the time.

So ......

That said I do empathise with both sides of this argument.
daniela florescu
2015-05-16 17:32:56 UTC
Permalink
In passing, I would like to observe that the title of the W3C Working Group being discussed is "XML Query Working Group" and the charter of that WG speaks of developing a query language for XML.
Jim,

please, this discussion starts to be repetitive and boring.

Stop running around the bush, too. Hypocrisy in technology is no good if you want to build useful; technology.

Simple hypocritical statement: “ we were not interested in querying JSON”. Huh. Yet you added JSON structures to it.

Why not Protocol Buffer ? Why Not SQL relations ? Why not CSV ? Why not Scala structures ? The world is full of useful “stuff”
.

Why adding JSON, if you were explicitly not interested by it !?

But obviously, I don’t expect an answer to this question. The answer is obvious. You think people can be so easily fooled!?
JSONiq is a very good technology developed specifically to integrate thorough JSON support alongside the XML Query facilities in XQuery.
You bet it is. I don’t design shitty technology, and a lot of thought (not only mine, obviously, but a whole excellent team) went into that design.
[[ In fact, many good things in XQuery come from me: the functional aspect of XQuery, the FLWOR expression, the element constructor with dynamic content,
the functions, the windowing, the composability of updates, etc, etc.]]

And, as a side, if you personally wouldn’t have told me “ We don’t need to hear your opinion” in a the middle of a W3C meeting, and you wouldn’t have allowed
Sharon Adler to kick me out of a W3C meeting by cutting my phone line (both illegal behaviors by the standards of W3C, but Liam didn’t seem to “notice"),
today XQuery would have had a very nicely designed Scripting Extension - similar to Stored Procedures in SQL, which would have enhanced the expressive
power of XQuery dramatically.


Like Zorba used to have, before Zorba team was killed, with your personal help, Jim.

So, you know, Jim, please stop the hypocrisy and the wooden language here. I think it’s better for everyone.

I honestly don’t care, except that I feel sorry that all the (hardly acquired!) experience of XQuery is lost on the JSON guys, and they’ll see another 15 years of tribulations,
and billions of dollars wasted on the industry.

Hasta la Vista,
Dana
However, as Mike Kay has stated several times, the goals of the XML Query WG were not identical to the goals of the group who designed JSONiq. Thus, given the primary focus of the XML Query WG and the goals of its participants, it is not surprising that the results of the WG's work differs from what the JSONiq developers might have hoped.
Hope this helps,
One of the morons
Post by Michael Kay
You can be assured that we had lively debates on this topic,
After “living” in the XQuery W3C working group for 15 years
.. I don’t doubt you had. I am just happy I was not there.
But, sorry, W3C took the wrong decision: in one instant with this decision, you lost the JSON community.
Is this not a variant of the Worse is Better argument (or did I not properly understand that essay...... probably).
The issue with many JSON people is that they don't seem to acknowledge the need for interoperability with XML so the utility of a "bilingual language" probably doesn't resonate.
Without wishing to appear informed can I ask a question.
Was the Crockfordesque faction represented in the composition of the WG or was it (of necessity) a bunch of XML people talking about how to support JSON?
--
========================================================================
Jim Melton --- Editor of ISO/IEC 9075-* (SQL) Phone: +1.801.942.0144
Chair, ISO/IEC JTC1/SC32 and W3C XML Query WG Fax : +1.801.942.3345
Oracle Corporation Oracle Email: jim dot melton at oracle dot com
1930 Viscounti Drive Alternate email: jim dot melton at acm dot org
Sandy, UT 84093-1063 USA Personal email: SheltieJim at xmission dot com
========================================================================
= Facts are facts. But any opinions expressed are the opinions =
= only of myself and may or may not reflect the opinions of anybody =
= else with whom I may or may not have discussed the issues at hand. =
========================================================================
Benito van der Zander
2015-05-18 15:23:41 UTC
Permalink
Hi,
if you query data from deep websites, you almost always end up with a
long get-link-download chain like:

doc("http://example.org") // a [ condition 1 ] / doc( @href) //a [
condition 2 ] / doc(@href) //a [ condition 3 ] / doc(@href) / ....

Half the query is a / doc( @href) filler.
It is really ugly, isn't it?

How about replacing / doc( @href) with a single map operator? (or more
general / doc((@href,@src,@action)[1])

Like ~

Looks much better:

doc("http://example.org") // a [ condition 1 ] ~ //a [ condition 2 ] ~
//a [ condition 3 ] ~ / ....

Or !!

doc("http://example.org") // a [ condition 1 ] !! //a [ condition 2 ] !!
//a [ condition 3 ] !! / ....

Does it not?



What do you think?

I guess the WG is not going to add it soon? Are any other ~ or !!
operators planed?
Because it would be a shame to add it in an implementation, and then get
XQuery 4, where there is a ~ operator that does something completely
different.

Best,
Benito

_______________________________________________
***@x-query.com
http://x-query.com/mailman/listinfo/talk
Michael Kay
2015-05-18 15:36:12 UTC
Permalink
Because it would be a shame to add it in an implementation, and then get XQuery 4, where there is a ~ operator that does something completely different.
Indeed, that is why XQuery does not allow a conformant implementation to add its own operators.

Michael Kay
Saxonica


_______________________________________________
***@x-query.com
http://x-query.com/mailman/
Benito van der Zander
2015-05-18 15:56:12 UTC
Permalink
Hi Michael,
Well, HTML defines its own semantics for all elements and attributes.
Post by Michael Kay
Indeed, that is why XQuery does not allow a conformant implementation to add its own operators.
Too late to care about that, when there are already JSONiq's constructors.


Cheers,
Benito
Post by Michael Kay
Because it would be a shame to add it in an implementation, and then get XQuery 4, where there is a ~ operator that does something completely different.
Indeed, that is why XQuery does not allow a conformant implementation to add its own operators.
Michael Kay
Saxonica
_______________________________________________
http://x-query.com/mailman/listinfo/talk
_______________________________________________
***@x-query.com
http://x-query.com
Michael Kay
2015-05-18 16:04:44 UTC
Permalink
Post by Benito van der Zander
Hi Michael,
Well, HTML defines its own semantics for all elements and attributes.
Yes, and that would be a possible way forward; though we’ve avoided providing XPath functions and operators that are specialized to a particular vocabulary in the past.
Post by Benito van der Zander
Post by Michael Kay
Indeed, that is why XQuery does not allow a conformant implementation to add its own operators.
Too late to care about that, when there are already JSONiq's constructors.
You can of course write implementations of languages that have the XQuery grammar as a subset, but they are not conformant XQuery implementations. Whether people care is another matter...

Michael Kay
Saxonica
_______________________________________________
***@x-query.com
http://x-query.com/
Michael Kay
2015-05-09 19:50:00 UTC
Permalink
That’s the standardisation process, unfortunately. It’s very rare for an existing design to be simply rubber-stamped by a standards committee. When SQL was standardized, ANSI didn’t simply rubber-stamp what IBM had done in System R.

What happens when you take a spec like JSONiq to a standards committee is that it gets scrutiny from a wider audience, some of whom want to address requirements that were outside the original scope. For some people, adding maps and arrays to XDM was more about increasing the expressive power of the language than specifically about supporting JSON; and the requirements for maps and arrays came from use cases other than JSON.
Post by daniela florescu
JSONiq is 3 years old, implemented by a variety of systems, and used in production in many places. And it was very well
designed: 2 years of VERY CAREFUL design by very good engineers went into the design of JSONiq.
I agree, it was an excellent piece of work, that did what it set out to do extremely well.
Post by daniela florescu
(a) it has a JSON-only subset that makes sense for the JSON-only community (who in general doesn’t want to see any shadow of angle brackets) and
(b) has the great advantage that doesn’t have the hated word “Xquery” in its name.
You may or may not be right, time will tell. I think it’s fairly inevitable that the user community addressed by the XQuery standards group will be the community that likes XQuery, not the community that hates it.
Post by daniela florescu
By not following JSONiq, by taking “slightly different” choices, or by not trying to extend it, XQuery 3.1 just broke the interoperability
in the NOSQL community, and this is what I consider a total lack of vision from the “leaders” of this community.
There were those on the committee who strongly defended the design choices that JSONiq had made. But no-one on the XQuery WG felt that the objective was to put a W3C stamp on work that had already been completed elsewhere.

Remember that JSONiq was not the only input to this work. The XSLT Working Group added maps to the XDM data model during 2010/2011 because they were needed for streaming use cases. During 2012 we become aware of the JSONiq work and attempted to bring the ideas into line; but we never considered dropping a feature (like using dates as keys in a map, or nodes as values) just because it wasn’t needed for JSON support.
Post by daniela florescu
***********************************************
* Maps can use any atomic value as the key, it does not have to be a string
* The members of an array are sequences, not necessarily items
* JSON’s null is represented as an empty sequence, not as a new atomic data type
* In XQ3.1, Map constructors use the syntax map{ a:b, c:d } rather than bare curlies
* XQ 3.1 introduces a lookup operator for maps and arrays: employee?name, or book?author?1
**********************************************
*********************************************
1. Was there any “mistake”, or “bug”, that was in JSONiq that those choices solved ?
I am not aware of any.
The differences were not motivated by fixing mistakes or bugs, but by addressing a wider range of use cases.
Post by daniela florescu
2. Are any of those choices “fundamental’, aka introduce a much larger expressive power, make the language much easier to use, etc.?
Anything fundamental in those “new” choices ?
Nope, they are just small, mostly insignificant, just enough itsy bitsy differences to confuse and irritate the entire NoSQL community.
I think the generalization of maps and arrays is rather fundamental, in that it increases the orthogonality of the data model. I cited the example of fn:apply(), which takes two arguments, a function and an array of arguments. Since each argument to a function is a sequence, we need a data structure which is an array (or sequence) of sequences, and limiting arrays to contain only items would have considerably reduced their power.
Post by daniela florescu
3. In case where a larger expressive power was desired, wasn’t it better to extend JSONiq in an upper compatible way, rather to take a different route ?
Most of these differences were in fact compatible. Some were not, like using the map{} keyword. That was a carefully debated decision, but I don’t think an argument of compatibility with JSONiq would have carried much weight: for better or for worse, standardizing JSONiq wasn’t the group’s charter.
Post by daniela florescu
4. Did XQuery 3.1 “committee” ever tried to contact the JSONiq community to try to find common solution that would eventual lead to a single, better language ?
I think that the JSONiq developers were well represented in the discussions and argued their case vigorously, and often successfully. The XSL WG made a lot of changes/concessions during this process: in fact, everyone did.
Post by daniela florescu
ALL decisions we took in the design of JSONiq were definitely not random, but VERY carefully thought out. You were not even curious of what those reason where,
No, that’s not true. I don’t know whether the “you” here is referring to me personally, or to the group as a whole, but in both cases, it’s not true. Many arguments were aired in great detail and were heard with patience by all concerned.
Post by daniela florescu
(and sometimes XQuery 3.1 got it totally wrong — I an write you a long email about THAT).
*********************************************
So: no bugs to be fixed, no new major technical advantages, and no desire to cooperate.
Getting to the point where we got to was not easy. There were passionate disagreements. The fact that we got a spec out is proof that everyone was trying to cooperate. Inevitably, not everyone will be happy with every decision that was made. Those who took part in the process are probably happier than those who did not, because they can see what the arguments were.
Post by daniela florescu
What conclusion do you get from here ?
I get a single conclusion: the XQuery 3.1 committee has a total lack of vision and leadership in the NoSQL world.
Well you might be right there. Standards groups don’t really do vision and leadership: they argue about braces and semicolons.
Post by daniela florescu
This can has two possible explanations: pure stupidity and lack of good will.
No, I think it’s simply the result of having a variety of perspectives. Standards groups bring people together who have different ideas about what it is important to achieve, and the result always involves compromises, which not everyone will be happy with.
Regards,
Michael Kay
Saxonica


_______________________________________________
***@x-query.com
http://x-query.com/ma
daniela florescu
2015-05-09 20:46:05 UTC
Permalink
Post by Michael Kay
Post by daniela florescu
I get a single conclusion: the XQuery 3.1 committee has a total lack of vision and leadership in the NoSQL world.
Well you might be right there. Standards groups don’t really do vision and leadership: they argue about braces and semicolons.
Michael,

I don’t get it.

You entire answer is telling me: "Standards groups don’t have (technical) vision and leadership."

That’s a HORRIBLE state of affairs, and I am surprised that this statement (that is indeed true) doesn’t shock you.

Then, how about they shut up and stop doing things, and imposing wrongly designed stuff on people. Or they could listen to people
who actually DO have technical vision and leadership !?

It seems that by “doing” things the XQuery WG hurts more then by being quiet, and close down.

In particular, XQuery 3.1 is a failure because has NOT been designed with the major requirement in mind they should have had, which is:

***************************

Make BOTH communities happy: XML and JSON, and help them work and integrate together.
Help the (younger) JSON community learn from some of the lessons the (older) XML community learned in the past 16 years about querying
and processing semi-strcutured data.

The world needs both, and the world needs them to work well together.

***************************


Why should some past choice made by XSLT in the past be more important then the goal I just stated — in big scheme of things !?


This is call short term thinking, and bad design.


Dana






_______________________________________________
***@x-query.com
http://x-q
Michael Kay
2015-05-09 21:51:24 UTC
Permalink
Post by daniela florescu
Post by Michael Kay
Post by daniela florescu
I get a single conclusion: the XQuery 3.1 committee has a total lack of vision and leadership in the NoSQL world.
Well you might be right there. Standards groups don’t really do vision and leadership: they argue about braces and semicolons.
Michael,
I don’t get it.
You entire answer is telling me: "Standards groups don’t have (technical) vision and leadership."
That’s a HORRIBLE state of affairs, and I am surprised that this statement (that is indeed true) doesn’t shock you.
Standards groups spend their time hammering out compromises to reconcile different visions. The result is very often a compromise. If they get it right, the result is a specification that meets a wider range of requirements and is acceptable to a wider community. But very few standards are noted for technical elegance.
Post by daniela florescu
Then, how about they shut up and stop doing things, and imposing wrongly designed stuff on people. Or they could listen to people
who actually DO have technical vision and leadership !?
Reminds me of my first XQuery group meeting: I was shocked by how many highly talented people there were with good ideas, all wanting to take it in different directions.
Post by daniela florescu
It seems that by “doing” things the XQuery WG hurts more then by being quiet, and close down.
***************************
Make BOTH communities happy: XML and JSON, and help them work and integrate together.
We definitely had people on the working group with the ambition to create a query language suitable for either XML or JSON as equal partners. Others felt that that was an unrealistic goal, and that XQuery should remain an XML query language with the ability to interoperate with JSON (and other formats) as second-class citizens. If you’re going to get anywhere in standards work, you have to appreciate that there’s going to be more than one vision, and that none of them are self-evidently right or wrong.
Post by daniela florescu
Why should some past choice made by XSLT in the past be more important then the goal I just stated — in big scheme of things !?
See above. You get inputs from lots of different directions, and they are all valid perspectives.

Michael Kay
Saxonica




_______________________________________________
***@x-query.com
daniela florescu
2015-05-09 22:00:54 UTC
Permalink
Post by Michael Kay
See above. You get inputs from lots of different directions, and they are all valid perspectives.
Com’ on MIchael. Stop running around the bush. I hate hypocrisy in technology.

Botton line is: as a result of what W3C did with XQuery 3.1, they created more harm them good overall for the industry.


And this: for both the XML community AND the JSON community.

For the XML community: they’ll be hated and avoided even more they used to be, and more and more isolated, and
For the JSON community: they’ll avoid anything related to XQuery like scary evil, which means that they’ll design silly query languages
by themselves (see Cassandra, see Mongo, see BigTable
.) for 15 years before finding some decent solution.


Great.

Thanks for your contribution.

Dana
Adam Retter
2015-05-09 23:42:09 UTC
Permalink
Post by daniela florescu
Botton line is: as a result of what W3C did with XQuery 3.1, they created
more harm them good overall for the industry.
And this: for both the XML community AND the JSON community.
For the XML community: they’ll be hated and avoided even more they used to
be, and more and more isolated, and
I don't understand your perspective at all.
I don't believe that XQuery is perfect, but then I don't believe that
any other programming or query language is either. Significantly
however we do have real XQuery 3.1 users (that were previously using
XQuery 3.0 and XQuery 1.0) publicly thanking us for the new features
of XQuery 3.1 that they are enjoying, here is one such recent thank
you - http://exist.markmail.org/thread/mb7jdspx5h3d67kj
Post by daniela florescu
For the JSON community: they’ll avoid anything related to XQuery like scary
evil, which means that they’ll design silly query languages
by themselves (see Cassandra, see Mongo, see BigTable….) for 15 years
before finding some decent solution.
JSON is important sure, but I don't believe it is the beginning and
the end of the Web and/or NoSQL. You mention Cassandra, but their
query language CQL appears to me to be inspired by SQL rather than
anything like JSONiq.

I really like JSONiq, I even started an implementation (unfinished) a
few years back. However, I have no sympathy for people or communities
that want to ignore a technology base because it is `scary evil`, I
don't buy into that as an argument, it just sounds like FUD; Serious
implementers of any language will always do their homework and learn
about the best and worst of their predecessors.

Regards Mongo, the only JSONiq implementation for that seems to be
from 28msec which you were heavily involved in I believe. Outside of
28msec and their partner work (IBM), apart from Xidel, I have not seen
any implementations of JSONiq. Certainly the NoSQL databases that you
mention, don't require a W3C stamped query language for them to
produce an implementation. I would be genuinely interested to know why
JSONiq was not more widely adopted? I really believed that JSONiq
would be snapped up very quickly by NoSQL JSON/BSON stores, Node.js
and others.

I think that if people want just a JavaScript query language for JSON
then why don't they just get/create an implementation of JSONiq in
JavaScript? Sure it could have been XQuery 3.1, but it's not... and
well... I think that is okay. XQuery 3.1 has its own use-cases and
purpose, it might not be as popular as JSON, but I don't see that as
an issue, they solve different (and sometimes similar) problems.
--
Adam Retter

skype: adam.retter
tweet: adamretter
http://www.adamretter.org.uk

_______________________________________________
daniela florescu
2015-05-09 23:44:18 UTC
Permalink
Adam,

sorry, will not respond this thread from now on. It’s a waste of my time.

Best
Dana
Post by Adam Retter
Post by daniela florescu
Botton line is: as a result of what W3C did with XQuery 3.1, they created
more harm them good overall for the industry.
And this: for both the XML community AND the JSON community.
For the XML community: they’ll be hated and avoided even more they used to
be, and more and more isolated, and
I don't understand your perspective at all.
I don't believe that XQuery is perfect, but then I don't believe that
any other programming or query language is either. Significantly
however we do have real XQuery 3.1 users (that were previously using
XQuery 3.0 and XQuery 1.0) publicly thanking us for the new features
of XQuery 3.1 that they are enjoying, here is one such recent thank
you - http://exist.markmail.org/thread/mb7jdspx5h3d67kj
Post by daniela florescu
For the JSON community: they’ll avoid anything related to XQuery like scary
evil, which means that they’ll design silly query languages
by themselves (see Cassandra, see Mongo, see BigTable….) for 15 years
before finding some decent solution.
JSON is important sure, but I don't believe it is the beginning and
the end of the Web and/or NoSQL. You mention Cassandra, but their
query language CQL appears to me to be inspired by SQL rather than
anything like JSONiq.
I really like JSONiq, I even started an implementation (unfinished) a
few years back. However, I have no sympathy for people or communities
that want to ignore a technology base because it is `scary evil`, I
don't buy into that as an argument, it just sounds like FUD; Serious
implementers of any language will always do their homework and learn
about the best and worst of their predecessors.
Regards Mongo, the only JSONiq implementation for that seems to be
from 28msec which you were heavily involved in I believe. Outside of
28msec and their partner work (IBM), apart from Xidel, I have not seen
any implementations of JSONiq. Certainly the NoSQL databases that you
mention, don't require a W3C stamped query language for them to
produce an implementation. I would be genuinely interested to know why
JSONiq was not more widely adopted? I really believed that JSONiq
would be snapped up very quickly by NoSQL JSON/BSON stores, Node.js
and others.
I think that if people want just a JavaScript query language for JSON
then why don't they just get/create an implementation of JSONiq in
JavaScript? Sure it could have been XQuery 3.1, but it's not... and
well... I think that is okay. XQuery 3.1 has its own use-cases and
purpose, it might not be as popular as JSON, but I don't see that as
an issue, they solve different (and sometimes similar) problems.
--
Adam Retter
skype: adam.retter
tweet: adamretter
http://www.adamretter.org.uk
_______________________________________________
***@x-query.com
http://x-query.com/mailman/listinf
William Candillon
2015-05-10 07:42:20 UTC
Permalink
Hi Adam,

To answer your last question, there is a typescript implementation of
JSONiq that started here: https://github.com/wcandillon/jsoniq.
It will be interesting to see the reception in the NoSQL and HTML5
community when this project will be a bit more mature.

It implements JSONiq updates and update composition on top of
IndexedDB. The end goal is to query/update IndexedDB and implement a
versioning system between IndexedDB and the cloud. Compiled queries
can run as standalone JavaScript programs: it's very lightweight.
It is tested against both browsers and nodejs.

This implementation is 100% organically grown. We started with an
XQuery parser in JavaScript for HTML5 editors. We then added static
analysis for errors and warnings (XQLint). XQLint is quite mature now
and the next logical step was to add a code generation step.

Kind regards,

William
Post by daniela florescu
Adam,
sorry, will not respond this thread from now on. It’s a waste of my time.
Best
Dana
Post by Adam Retter
Post by daniela florescu
Botton line is: as a result of what W3C did with XQuery 3.1, they created
more harm them good overall for the industry.
And this: for both the XML community AND the JSON community.
For the XML community: they’ll be hated and avoided even more they used to
be, and more and more isolated, and
I don't understand your perspective at all.
I don't believe that XQuery is perfect, but then I don't believe that
any other programming or query language is either. Significantly
however we do have real XQuery 3.1 users (that were previously using
XQuery 3.0 and XQuery 1.0) publicly thanking us for the new features
of XQuery 3.1 that they are enjoying, here is one such recent thank
you - http://exist.markmail.org/thread/mb7jdspx5h3d67kj
Post by daniela florescu
For the JSON community: they’ll avoid anything related to XQuery like scary
evil, which means that they’ll design silly query languages
by themselves (see Cassandra, see Mongo, see BigTable….) for 15 years
before finding some decent solution.
JSON is important sure, but I don't believe it is the beginning and
the end of the Web and/or NoSQL. You mention Cassandra, but their
query language CQL appears to me to be inspired by SQL rather than
anything like JSONiq.
I really like JSONiq, I even started an implementation (unfinished) a
few years back. However, I have no sympathy for people or communities
that want to ignore a technology base because it is `scary evil`, I
don't buy into that as an argument, it just sounds like FUD; Serious
implementers of any language will always do their homework and learn
about the best and worst of their predecessors.
Regards Mongo, the only JSONiq implementation for that seems to be
from 28msec which you were heavily involved in I believe. Outside of
28msec and their partner work (IBM), apart from Xidel, I have not seen
any implementations of JSONiq. Certainly the NoSQL databases that you
mention, don't require a W3C stamped query language for them to
produce an implementation. I would be genuinely interested to know why
JSONiq was not more widely adopted? I really believed that JSONiq
would be snapped up very quickly by NoSQL JSON/BSON stores, Node.js
and others.
I think that if people want just a JavaScript query language for JSON
then why don't they just get/create an implementation of JSONiq in
JavaScript? Sure it could have been XQuery 3.1, but it's not... and
well... I think that is okay. XQuery 3.1 has its own use-cases and
purpose, it might not be as popular as JSON, but I don't see that as
an issue, they solve different (and sometimes similar) problems.
--
Adam Retter
skype: adam.retter
tweet: adamretter
http://www.adamretter.org.uk
_______________________________________________
http://x-query.com/mailman/listinfo/talk
_______________________________________________
***@x-query.
Adam Retter
2015-05-10 08:27:02 UTC
Permalink
Post by William Candillon
To answer your last question, there is a typescript implementation of
JSONiq that started here: https://github.com/wcandillon/jsoniq.
It will be interesting to see the reception in the NoSQL and HTML5
community when this project will be a bit more mature.
It implements JSONiq updates and update composition on top of
IndexedDB. The end goal is to query/update IndexedDB and implement a
versioning system between IndexedDB and the cloud. Compiled queries
can run as standalone JavaScript programs: it's very lightweight.
It is tested against both browsers and nodejs.
That's awesome and exactly the sort of good fit for JSONiq that I
expected to be adopted by the JavaScript community.

I wondered in terms of JSONiq support whether it is possible to do
without IndexedDB, i.e. just be able to query JSON files and/or JSON
coming in over the http (via AJAX and WebSockets etc)?

Out of interest, regards Node.js, what do you use for the IndexedDB?
Post by William Candillon
This implementation is 100% organically grown. We started with an
XQuery parser in JavaScript for HTML5 editors. We then added static
analysis for errors and warnings (XQLint). XQLint is quite mature now
and the next logical step was to add a code generation step.
I wish you all the best with this project. I hope it gets the
attention it deserves from the JavaScript community. I will be
interested to see how it goes...
--
Adam Retter

skype: adam.retter
tweet: adamretter
http://www.adamretter.org.uk
_______________________________________________
***@x-query.com
http://x-query.com/mailman/listinfo/talk
William Candillon
2015-05-10 08:39:44 UTC
Permalink
Post by Adam Retter
Post by William Candillon
To answer your last question, there is a typescript implementation of
JSONiq that started here: https://github.com/wcandillon/jsoniq.
It will be interesting to see the reception in the NoSQL and HTML5
community when this project will be a bit more mature.
It implements JSONiq updates and update composition on top of
IndexedDB. The end goal is to query/update IndexedDB and implement a
versioning system between IndexedDB and the cloud. Compiled queries
can run as standalone JavaScript programs: it's very lightweight.
It is tested against both browsers and nodejs.
That's awesome and exactly the sort of good fit for JSONiq that I
expected to be adopted by the JavaScript community.
I wondered in terms of JSONiq support whether it is possible to do
without IndexedDB, i.e. just be able to query JSON files and/or JSON
coming in over the http (via AJAX and WebSockets etc)?
The architecture is completely decoupled from IndexedDB.
Querying/Updating IndexedDB is simply the closest use case for us.
Post by Adam Retter
Out of interest, regards Node.js, what do you use for the IndexedDB?
Post by William Candillon
This implementation is 100% organically grown. We started with an
XQuery parser in JavaScript for HTML5 editors. We then added static
analysis for errors and warnings (XQLint). XQLint is quite mature now
and the next logical step was to add a code generation step.
I wish you all the best with this project. I hope it gets the
attention it deserves from the JavaScript community. I will be
interested to see how it goes...
Thanks Adam!
Post by Adam Retter
--
Adam Retter
skype: adam.retter
tweet: adamretter
http://www.adamretter.org.uk
_______________________________________________
***@x-query.com
http://x-query.com/mailman/listinfo/talk
Adam Retter
2015-05-10 09:37:44 UTC
Permalink
Post by William Candillon
Post by Adam Retter
I wondered in terms of JSONiq support whether it is possible to do
without IndexedDB, i.e. just be able to query JSON files and/or JSON
coming in over the http (via AJAX and WebSockets etc)?
The architecture is completely decoupled from IndexedDB.
Querying/Updating IndexedDB is simply the closest use case for us.
Nice. Hopefully in a few months when things let up a bit I will have
some time to take it for a spin.
--
Adam Retter

skype: adam.retter
tweet: adamretter
http://www.adamretter.org.uk
_______________________________________________
***@x-query.com
http://x-query.com/mailman/listinfo/talk
Loading...