[xquery-talk] Izzit Bcos I is functional?

[Ihe Onwuka]

Probably the wrong place to ask but what are the prime factors behind the
resistance to adopt XQuery or it's derivatives thereof as a technology for
querying and manipulating semi-structured data.

[Michael Kay]

Yes, it’s the wrong place to ask. But my guess would be:

(a) XSLT is well entrenched for doing many of the jobs that could also be done with XQuery.

(b) Lots of programmers, given a new job to do, prefer to use the tools they know, such as Java, PHP, or C#, rather than learning a specialized tool

(c) In the database arena, there is strong corporate resistance to increasing the variety of database technology in use within an enterprise.

(d) Resistance in some quarters to using XML as the automatic solution for long-term storage of “semi-structured data”.

These are actually all (at least in some cases) legitimate objections; but they are all compounded by fear of the unknown (which is also legitimate).

Don’t assume that failure to adopt XQuery is because of resistance to adopting XQuery. The reason I haven’t adopted Erlang isn’t because I’m resistant to it; I’ve just never had a requirement that caused me to think “It’s time I considered Erlang as the solution to this problem”. There are an awful lot of good technologies competing for attention out there and no one has time to look at them all.

Michael Kay
Saxonica

Probably the wrong place to ask but what are the prime factors behind the resistance to adopt XQuery or it's derivatives thereof as a technology for querying and manipulating semi-structured data.
_______________________________________________
http://x-query.com/mailman/listinfo/talk

_______________________________________________
t

[daniela florescu]

Ihe,

based on my own personal experience, I could give you the long version of the answer and the short version
of the answer.

I’ll start with the short one.

And I will start with the NON-reason for the non-sucess of Xquery.

1. No, it’s not because it’s functional.

Even though, because it’s functional, it will be restricted to be used only by people with CS degrees, and not by
random Joes and Janes who write web sites. The way it is designed it is intended to make a population of educated programmers
ETREMELY efficient, and NOT to increase the total number of developers to hundreds of millions.

When being reproached this fact in the past, my answer was always the same: building a database application should not be for the uneducated.
It’s like building a 30 story building, you don’t do that without a architect ad a structural engineer.
E.g. if you want to eradicate a grave neurological disease, you don’t lower the bar to allow anyone from the street to perform a neurosurgery,
you just make the existing neurosurgents more productive.

And BTW, XQuery (like any programming language in 2015) should not be written by hand, by mostly automatically generated by tools, so at the end,
who cares if it is functional or not.

2. It’s not because academia doesn’t pay attention.

That’s not true. Almost every database class I know finishes with teaching the students XQuery. I taught full XML/XQuery classes myself in both Stanford and Berkeley
and the students loved it. It is true that most database professors themselves don’t understand XML and XQuery, but that’s another story……

==============

Here would be my FIRST real reason:

1. XQuery CANNOT be more successful then the problem that it tries to solve, which is XML processing.

XML itself is not successful.Period.
There is no money in XML. Period.

There is a USE CASE in XML as documents, but not enough money in this market.

And XML as data is a total flop. XML is hated and avoided by the developers like hell. And that, for good reasons.

So, why wouldn’t they use XQuery, when they don’t want to see the face of XML in the first place !?

So, you see, it would be unreasonable to expect that XQuery is successful in places where XML is hated.

E.G. MarkLogic after 14 years of existence barely managed to pass 100M revenue. DatasTax after 3 years
of existence is at more then 300M revenue (and less investment from VCs).

It’s … XML vs. JSON. Documents vs. data.

So…. I think it is simply a question of …. there is no market for XML ……(aka no enough MONEY in the market).


(there are plenty of other reasons, of course, but I think this is the main one..)

=============

The only way for the ideas behind XQuery to become successful is trough JSON and a language like JSONiq.

Because there IS enough money in the market behind JSON…

That’s my short answer.

I can send you the longer answer, maybe later.

Best
Dana

Post by Michael Kay
(a) XSLT is well entrenched for doing many of the jobs that could also be done with XQuery.
(b) Lots of programmers, given a new job to do, prefer to use the tools they know, such as Java, PHP, or C#, rather than learning a specialized tool
(c) In the database arena, there is strong corporate resistance to increasing the variety of database technology in use within an enterprise.
(d) Resistance in some quarters to using XML as the automatic solution for long-term storage of “semi-structured data”.
These are actually all (at least in some cases) legitimate objections; but they are all compounded by fear of the unknown (which is also legitimate).
Don’t assume that failure to adopt XQuery is because of resistance to adopting XQuery. The reason I haven’t adopted Erlang isn’t because I’m resistant to it; I’ve just never had a requirement that caused me to think “It’s time I considered Erlang as the solution to this problem”. There are an awful lot of good technologies competing for attention out there and no one has time to look at them all.
Michael Kay
Saxonica

Probably the wrong place to ask but what are the prime factors behind the resistance to adopt XQuery or it's derivatives thereof as a technology for querying and manipulating semi-structured data.
_______________________________________________
http://x-query.com/mailman/listinfo/talk

_______________________________________________
http://x-query.com/mailman/listinfo/talk

_______________________________________________
***@x-query.com
http://x-query.com/

[Ihe Onwuka]

Post by daniela florescu
Ihe,
based on my own personal experience, I could give you the long version of
the answer and the short version
of the answer.
I’ll start with the short one.
And I will start with the NON-reason for the non-sucess of Xquery.
1. No, it’s not because it’s functional.
Even though, because it’s functional, it will be restricted to be used
only by people with CS degrees, and not by
random Joes and Janes who write web sites. The way it is designed it is
intended to make a population of educated programmers
ETREMELY efficient, and NOT to increase the total number of developers to
hundreds of millions.
When being reproached this fact in the past, my answer was always the
same: building a database application should not be for the uneducated.
It’s like building a 30 story building, you don’t do that without a
architect ad a structural engineer.
E.g. if you want to eradicate a grave neurological disease, you don’t
lower the bar to allow anyone from the street to perform a neurosurgery,
you just make the existing neurosurgents more productive.
And BTW, XQuery (like any programming language in 2015) should not be
written by hand, by mostly automatically generated by tools, so at the end,
who cares if it is functional or not.
2. It’s not because academia doesn’t pay attention.
That’s not true. Almost every database class I know finishes with teaching
the students XQuery. I taught full XML/XQuery classes myself in both
Stanford and Berkeley
and the students loved it. It is true that most database professors
themselves don’t understand XML and XQuery, but that’s another story



I'm not sure about that. I know Harvard does but that is part of a web
programming course.

http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-830-database-systems-fall-2010/readings/
http://ocw.mit.edu/courses/civil-and-environmental-engineering/1-264j-database-internet-and-systems-integration-technologies-fall-2013/lecture-notes-exercises/

MIT doesn't, neither does either of the universities I went to.

Post by daniela florescu
==============
1. XQuery CANNOT be more successful then the problem that it tries to
solve, which is XML processing.
XML itself is not successful.Period.
There is no money in XML. Period.
There is a USE CASE in XML as documents, but not enough money in this market.
And XML as data is a total flop. XML is hated and avoided by the
developers like hell. And that, for good reasons.

What are they. Alot of the ones I have read aren't true and seem to be
based on a lack of knowledge about XML

Post by daniela florescu
So, why wouldn’t they use XQuery, when they don’t want to see the face of
XML in the first place !?
So, you see, it would be unreasonable to expect that XQuery is successful
in places where XML is hated.
E.G. MarkLogic after 14 years of existence barely managed to pass 100M
revenue. DatasTax after 3 years
of existence is at more then 300M revenue (and less investment from VCs).
It’s 
 XML vs. JSON. Documents vs. data.

Y'see I reckon that if JSON was deployed in many of the domains where XML
is, it too would be hated.

Post by daniela florescu
So
. I think it is simply a question of 
. there is no market for XML


(aka no enough MONEY in the market).
(there are plenty of other reasons, of course, but I think this is the main one..)
=============
The only way for the ideas behind XQuery to become successful is trough
JSON and a language like JSONiq.
Because there IS enough money in the market behind JSON

That’s my short answer.
I can send you the longer answer, maybe later.

Eagerly awaited.

[daniela florescu]

I'm not sure about that. I know Harvard does but that is part of a web programming course.
http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-830-database-systems-fall-2010/readings/ <http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-830-database-systems-fall-2010/readings/>
http://ocw.mit.edu/courses/civil-and-environmental-engineering/1-264j-database-internet-and-systems-integration-technologies-fall-2013/lecture-notes-exercises/ <http://ocw.mit.edu/courses/civil-and-environmental-engineering/1-264j-database-internet-and-systems-integration-technologies-fall-2013/lecture-notes-exercises/>

Look inside the slides of the database classes
 you’ll see XQuery almost everywhere (at the end of the class
).

Especially that Stanford does it, and most database professors copy the slides from Stanford.
https://lagunita.stanford.edu/courses/DB/XPath/SelfPaced/courseware/ch-querying_xml/seq-vid-xquery_introduction/ <https://lagunita.stanford.edu/courses/DB/XPath/SelfPaced/courseware/ch-querying_xml/seq-vid-xquery_introduction/>

(not that the Stanford professors understand much from XML and XQuery 
 but that’s a whole new discussion
)

MIT doesn't, neither does either of the universities I went to.

That’s because Stonebreaker is there, and he wouldn’t be caught dead teaching XML. I’ll see if I can make him change his mind
about JSON
. :-)

The fact that sir TBL is in MIT too doesn’t help at all either, given the fact that he was “convinced” by a bunch of good willing people that XML is
evil. (and the semantic web will cure the hunger of the world and the cancer for sure
)

What are they. Alot of the ones I have read aren't true and seem to be based on a lack of knowledge about XML

I think I KNOW something about XML — not that I enjoyed learning it, but I HAD to :-)

Want a quick list ? processing instructions, weirdo parsing rules that are inherited from the 1950’s SGML, weirdo design
of namespaces, XML Schema anyone !? nillable anyone !? Some early design of Xpath 1.0 (no reserved keywords, semantics of =)
I’ll stop here.

Only those and that would stop me (as a database person) right there to use XML as a format for data.

Y'see I reckon that if JSON was deployed in many of the domains where XML is, it too would be hated.

Yes, but less.

People would STILL have to face the fact that they cannot process JSON with their beloved SQL
. but at least they wouldn’t
add insult to injury by adding the weird things I listed above.

The bitter pill would be somehow easier to swallow.

The database people accept with open arms JSOn right now because they don’t REALLY understand what JSON is.

Most of them still look at JSON as a nicer syntax then CSV
..they think it’s flat, or a “little” nested.

The shock will hit them later on :-))))

Best regards
Dana

[daniela florescu]

Post by daniela florescu
Especially that Stanford does it, and most database professors copy the slides from Stanford.
https://lagunita.stanford.edu/courses/DB/XPath/SelfPaced/courseware/ch-querying_xml/seq-vid-xquery_introduction/ <https://lagunita.stanford.edu/courses/DB/XPath/SelfPaced/courseware/ch-querying_xml/seq-vid-xquery_introduction/>
(not that the Stanford professors understand much from XML and XQuery 
 but that’s a whole new discussion
)

Just a proof that, even if they DO teach XQuery, Stanford professors don’t understand what their are reaching.

Just look at the second bullet on the slide: “each expression operates and returns a sequence of ELEMENTS”.

:-))))

Yep, XQuery lacks a little bit of good tutorials and good books, and good teaching material.

Marketing, in general.

Best
Dana

[Maurice van Keulen]

Just to confirm, at the University of Twente (Netherlands), we teach
XML/XQuery alongside Relational/SQL to the first year bachelor students
computer science. This is their 'foundations of databases' course. One
of the main reasons is to teach them that there is structured,
semi-structured and unstructured data which are best processed with
different technologies. Regarding XQuery, my end goal is that they can
develop their first web services with RESTXQ (using BaseX), just to make
them realize how much more productive they can become when embracing
such technology.

Regards,
Maurice van Keulen

Post by Ihe Onwuka
I'm not sure about that. I know Harvard does but that is part of a
web programming course.
http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-830-database-systems-fall-2010/readings/
http://ocw.mit.edu/courses/civil-and-environmental-engineering/1-264j-database-internet-and-systems-integration-technologies-fall-2013/lecture-notes-exercises/

Look inside the slides of the database classes… you’ll see XQuery
almost everywhere (at the end of the class…).
Especially that Stanford does it, and most database professors copy
the slides from Stanford.
https://lagunita.stanford.edu/courses/DB/XPath/SelfPaced/courseware/ch-querying_xml/seq-vid-xquery_introduction/
(not that the Stanford professors understand much from XML and XQuery
… but that’s a whole new discussion…)

Post by Ihe Onwuka
MIT doesn't, neither does either of the universities I went to.

That’s because Stonebreaker is there, and he wouldn’t be caught dead
teaching XML. I’ll see if I can make him change his mind
about JSON…. :-)
The fact that sir TBL is in MIT too doesn’t help at all either, given
the fact that he was “convinced” by a bunch of good willing people
that XML is
evil. (and the semantic web will cure the hunger of the world and the
cancer for sure…)

Post by Ihe Onwuka
What are they. Alot of the ones I have read aren't true and seem to
be based on a lack of knowledge about XML

I think I KNOW something about XML — not that I enjoyed learning it,
but I HAD to :-)
Want a quick list ? processing instructions, weirdo parsing rules that
are inherited from the 1950’s SGML, weirdo design
of namespaces, XML Schema anyone !? nillable anyone !? Some early
design of Xpath 1.0 (no reserved keywords, semantics of =)
I’ll stop here.
Only those and that would stop me (as a database person) right there
to use XML as a format for data.

Post by Ihe Onwuka
Y'see I reckon that if JSON was deployed in many of the domains where
XML is, it too would be hated.

Yes, but less.
People would STILL have to face the fact that they cannot process JSON
with their beloved SQL…. but at least they wouldn’t
add insult to injury by adding the weird things I listed above.
The bitter pill would be somehow easier to swallow.
The database people accept with open arms JSOn right now because they
don’t REALLY understand what JSON is.
Most of them still look at JSON as a nicer syntax then CSV…..they
think it’s flat, or a “little” nested.
The shock will hit them later on :-))))
Best regards
Dana
_______________________________________________
http://x-query.com/mailman/listinfo/talk

[Ihe Onwuka]

Maurice,

Can you relate, or is it different in the Netherlands. I can't remember why
I got it from but it is very well written.

I'm a university lecturer in Computer Science, I teach a second year module
in programming. I've spent my Christmas "holidays" marking the end-of-term
test I set for my students. Every year it's the same - no more than a third
of them are showing the sort of ability I would want in anyone doing a
coding job. One-third of them are so poor at programming that one would be
surprised to hear they had spent more than a couple of weeks supposedly
learning about it, never mind half-way through a degree in it. This is the
hidden issue, which academics moan about in private but don't like to say
in public. It's not just me or where I teach, the same occurs everywhere.
If you really test them on decent programming skills, you get a huge
failure rate. In this country it's thought bad to fail students, so mostly
we find ways of getting them through even though they don't really have the
skills. My advice to any employer looking for real programming skills is
don't just look at the degree class, look at the module grades, and don't
take on anyone whose 2i or 1st has been gained by good grades in other
modules balancing weak grade in the programming modules. But even with the
programming modules, check out how the assessment was done. One issue does
seem to be natural ability - some have got it, some haven't. We've not
found a good way to work out who has it when we recruit the students, and
league table pressure now means we are forced to recruit the students with
the highest A-level grades regardless of anything else, and A-level grades
certainly are NOT a good test of natural ability for coding. There's some
correlation with the grade in A-level Maths, that's about it. However,
these days when all the pressure is to recruit high grade students because
that pushes us up the league tables, central admin won't let us take on
applicants with reasonable A-level maths and outside interests which
suggest genuine coding ability rather than applicants with just higher
grades in other subjects. But another issue is lack of student discipline.

Too many of them just don't realise how to learn this stuff properly, you
have to practice, practice, practice. Too many of them are stuck in the
mentality that to do well all that is needed is to "revise" in the week
before the exam. Unfortunately, we have a culture which puts forward the
impression that tests and exams are "memory dumps". I keep telling my
students it isn't like that for our subject, they don't believe me. I see
this in their test and exam answers, where you so often see something which
completely misses the point written down because it's something they've
memorised and they've dumped it on paper in response to a trigger word in
the question. Note also, if the assessment is more through coursework than
exams, a large proportion of what is submitted will not be the students'
own work. It may not be actual copying, but it is often "We discussed it
together" i.e. one of them did the work, and the others made cosmetic
fiddles and claimed they were just helped by the discussion. Or some friend
or relative did it for them, etc. So I would say, if assessment is not
largely exam based, don't take them on even if they do have high grades in
the programming modules. This is also why I have no faith in "MOOC"s, as
most of the seem to have assessment based on memorisation and multiple
choice rather than deep analysis of real skills. Sure, you can automate
marking of really simple introductory programming exercises, but anything
beyond 1st year coding, you can't, because automated marking won't test
subtle issues such as good style or solving a problem in a particular way.
I set tests and exams with an emphasis on writing actual code and showing
deep understanding, but it doesn't make me popular, and it doesn't lead to
a high pass rate. Many others succumb to the pressure, and set tests and
exams with questions which are multiple-choice and centre on memorising
definitions rather than solving coding problems. That makes it MUCH easier
to mark, and you get a higher pass rate as well. If I had done that I
wouldn't have my wife nagging me "It's Christmas - why are you doing
work?", and I wouldn't have my boss nagging me "Why does your module have
such a high failure rate?" and I wouldn't have my student nagging me "Why
are you so tough on us?".

Post by Maurice van Keulen
Just to confirm, at the University of Twente (Netherlands), we teach
XML/XQuery alongside Relational/SQL to the first year bachelor students
computer science. This is their 'foundations of databases' course. One of
the main reasons is to teach them that there is structured, semi-structured
and unstructured data which are best processed with different technologies.
Regarding XQuery, my end goal is that they can develop their first web
services with RESTXQ (using BaseX), just to make them realize how much more
productive they can become when embracing such technology.
Regards,
Maurice van Keulen
I'm not sure about that. I know Harvard does but that is part of a web programming course.
http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-830-database-systems-fall-2010/readings/
http://ocw.mit.edu/courses/civil-and-environmental-engineering/1-264j-database-internet-and-systems-integration-technologies-fall-2013/lecture-notes-exercises/
Look inside the slides of the database classes
 you’ll see XQuery almost
everywhere (at the end of the class
).
Especially that Stanford does it, and most database professors copy the
slides from Stanford.
https://lagunita.stanford.edu/courses/DB/XPath/SelfPaced/courseware/ch-querying_xml/seq-vid-xquery_introduction/
(not that the Stanford professors understand much from XML and XQuery 

but that’s a whole new discussion
)
MIT doesn't, neither does either of the universities I went to.
That’s because Stonebreaker is there, and he wouldn’t be caught dead
teaching XML. I’ll see if I can make him change his mind
about JSON
. :-)
The fact that sir TBL is in MIT too doesn’t help at all either, given
the fact that he was “convinced” by a bunch of good willing people that XML
is
evil. (and the semantic web will cure the hunger of the world and the
cancer for sure
)
What are they. Alot of the ones I have read aren't true and seem to be
based on a lack of knowledge about XML
I think I KNOW something about XML — not that I enjoyed learning it, but I HAD to :-)
Want a quick list ? processing instructions, weirdo parsing rules that
are inherited from the 1950’s SGML, weirdo design
of namespaces, XML Schema anyone !? nillable anyone !? Some early design
of Xpath 1.0 (no reserved keywords, semantics of =)
I’ll stop here.
Only those and that would stop me (as a database person) right there to
use XML as a format for data.
Y'see I reckon that if JSON was deployed in many of the domains where
XML is, it too would be hated.
Yes, but less.
People would STILL have to face the fact that they cannot process JSON
with their beloved SQL
. but at least they wouldn’t
add insult to injury by adding the weird things I listed above.
The bitter pill would be somehow easier to swallow.
The database people accept with open arms JSOn right now because they
don’t REALLY understand what JSON is.
Most of them still look at JSON as a nicer syntax then CSV
..they think
it’s flat, or a “little” nested.
The shock will hit them later on :-))))
Best regards
Dana
--
-----------------------------------------------------------------------
Dr.ir. M. van Keulen - Associate Professor Data Management Technology
Univ of Twente, Dept of EEMCS, POBox 217, 7500 AE Enschede, Netherlands
Room: Zilverling 2013, WWW: http://www.cs.utwente.nl/~keulen

[G. Ken Holman]

I'm not a university lecture, but I do (did) teach hands-on XSLT and XQuery.

Every year it's the same - no more than a third of them are showing
the sort of ability I would want in anyone doing a coding job.

I can testify to that in the students who showed up at my classes.

But another issue is lack of student discipline.
Too many of them just don't realise how to learn this stuff
properly, you have to practice, practice, practice.

What I learned at the University of Waterloo was how to learn, not
what to learn. For example, every week in the programming languages
class the assignment due was in a different programming
language. The point wasn't learning the programming language, but to
learn how to learn a new programming language. As Ihe says, that
takes a lot of practice, and learning a new programming language
every week provided exactly that: practice in learning. And I'm
pleased (relieved) to say I've never had to use SNOBOL in the real world.

Too many of them are stuck in the mentality that to do well all that
is needed is to "revise" in the week before the exam.

I graduated in 1981, SGML was finalized in 1983 and I didn't start
working with it until 1989. So there was no opportunity to learn
markup when I was in university (though perhaps I should have paid a
bit more attention in the computer grammar class). Whatever students
cram in for the exam is likely not going to be useful years
later. But how they learned (by not cramming) will hold them in good
stead regardless of the problems their employers are asking them to solve.

If I had done that I wouldn't have my wife nagging me "It's
Christmas - why are you doing work?", and I wouldn't have my boss
nagging me "Why does your module have such a high failure rate?" and
I wouldn't have my student nagging me "Why are you so tough on us?".

I bet not so tough as an employer with a customer deadline!

I acknowledge this may be seen as veering off topic, but as I exit
the scene I am not holding a lot of promise that these wonderful
technologies are going to be wielded at all well in the foreseeable future.

In my (limited) perspective I fail to see what stepping back to
accommodate JSON is bringing to the party. I liked what I read
earlier in this thread (or was it another?) that JSON is only
ephemeral and XML can be persistent. Fine by me, then, if JSON is
solving a different problem than what I've been solving. But I feel
it falls short in addressing the problems that XML addresses, so I
worry when I read of people replacing XML with JSON for everything.

I think the industry needs to listen to people like Maurice and Ihe
and focus not on changing the technologies that are there, but to
focus on getting them better applied as they are. That is where I
think energy needs to be spent.

. . . . . . . . . Ken


--
Check our site for free XML, XSLT, XSL-FO and UBL developer resources |
Free 5-hour lecture: http://www.CraneSoftwrights.com/links/video.htm |
Crane Softwrights Ltd. http://www.CraneSoftwrights.com/q/ |
G. Ken Holman mailto:***@CraneSoftwrights.com |
Google+ profile: http://plus.google.com/+GKenHolman-Crane/about |
Legal business disclaimers: http://www.CraneSoftwrights.com/legal |


---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus

_______________________________________________
***@x-query.com
http://x-query.com/mailman/listinfo/talk

[Ihe Onwuka]

On Fri, Jun 19, 2015 at 8:07 AM, G. Ken Holman <

I think the industry needs to listen to people like Maurice and Ihe and
focus not on changing the technologies that are there, but to focus on
getting them better applied as they are. That is where I think energy
needs to be spent.

I am being given too much credit here. I am not a lecturer, the piece was
written by somebody else, I just pasted it here, but of course I heartily
agree with what it says and what you said also.

[Ihe Onwuka]

Post by Ihe Onwuka
What are they. Alot of the ones I have read aren't true and seem to be
based on a lack of knowledge about XML
I think I KNOW something about XML — not that I enjoyed learning it, but I HAD to :-)
Want a quick list ? processing instructions, weirdo parsing rules that are
inherited from the 1950’s SGML, weirdo design
of namespaces, XML Schema anyone !? nillable anyone !? Some early design
of Xpath 1.0 (no reserved keywords, semantics of =)
I’ll stop here.
Only those and that would stop me (as a database person) right there to
use XML as a format for data.

But you can pretty much model the semantic equivalent of JSON data without
having to use a Schema, namespace, PI or any traverse any other of these
treacherous terrains. Call it XML the Good Parts (Hmmmm where did I get
that idea from). JSON is a very expensive over-reaction. There was no need
to invent another format and then make in non-interoperable with XML.

[Michael Kay]

But you can pretty much model the semantic equivalent of JSON data without having to use a Schema, namespace, PI or any traverse any other of these treacherous terrains. Call it XML the Good Parts (Hmmmm where did I get that idea from). JSON is a very expensive over-reaction. There was no need to invent another format and then make in non-interoperable with XML.

Actually, I don’t agree. The cost of doing simple things with XML, like configuration files, is far too high (because of all the other things it is capable of that you don’t need for such cases) and there was a very real need for something simpler for that kind of application. Restricting yourself to a subset of XML doesn’t greatly reduce the cost of writing code to process it, you still have to use the same APIs. Perhaps it could have been achieved with some kind of MicroXML, but as a community, we failed to deliver that.

Michael Kay
Saxonica
_______________________________________________
***@x-query.com
http://x-query.com/mailm

[Ihe Onwuka]

Post by Ihe Onwuka
But you can pretty much model the semantic equivalent of JSON data

without having to use a Schema, namespace, PI or any traverse any other of
these treacherous terrains. Call it XML the Good Parts (Hmmmm where did I
get that idea from). JSON is a very expensive over-reaction. There was no
need to invent another format and then make in non-interoperable with XML.
Actually, I don’t agree. The cost of doing simple things with XML, like
configuration files, is far too high (because of all the other things it is
capable of that you don’t need for such cases) and there was a very real
need for something simpler for that kind of application.

But that is not the same as saying that the replacement of XML with JSON as
a format for generic data interchange was warranted, and making it not
interoperable with XML was spectacularly short sighted. Now true that
probably wasn't the intended use case. Sadly the lack of interoperability
and the consequent inability to deploy a common set of tools is probably
seen as a virtue in that community.

Restricting yourself to a subset of XML doesn’t greatly reduce the cost of
writing code to process it, you still have to use the same APIs.

When they are done implementing everything with JSON they will find they
will have something that will look just like XML, have most if not all of
it's atttendant overheads but be even more expensive as it will have been
re-implemented from scratch.

I'm just waiting for the next straight faced instalment.

What NOSQL needs now is a transformation language.

[daniela florescu]

Post by Michael Kay

But you can pretty much model the semantic equivalent of JSON data without having to use a Schema, namespace, PI or any traverse any other of these treacherous terrains. Call it XML the Good Parts (Hmmmm where did I get that idea from). JSON is a very expensive over-reaction. There was no need to invent another format and then make in non-interoperable with XML.

Actually, I don’t agree. The cost of doing simple things with XML, like configuration files, is far too high (because of all the other things it is capable of that you don’t need for such cases) and there was a very real need for something simpler for that kind of application. Restricting yourself to a subset of XML doesn’t greatly reduce the cost of writing code to process it, you still have to use the same APIs. Perhaps it could have been achieved with some kind of MicroXML, but as a community, we failed to deliver that.

I agree with Michael here. There was a need, and the world satisfied it. That’s all.

But in any case, the world could not care less about what WE think at this point.

Both XML and JSON are here to stay. There are good reasons for each one of them.

So, ideally, in the future:

(1) the two technologies will not be developed completely independently (users will need to deal with both in the
same time, so that would be a terribly expensive exercise for the industry as a whole) and

(2) JSON people will learn something from what has been developed in the XML world , and they won’t start from scratch , aka
from were XML started in 1996.

We can dream, can’t we !?? :-)

Best,
Dana





_______________________________________________
***@x-query.com
htt

[Ihe Onwuka]

Post by Ihe Onwuka
But you can pretty much model the semantic equivalent of JSON data

without having to use a Schema, namespace, PI or any traverse any other of
these treacherous terrains. Call it XML the Good Parts (Hmmmm where did I
get that idea from). JSON is a very expensive over-reaction. There was no
need to invent another format and then make in non-interoperable with XML.

Actually, I don’t agree. The cost of doing simple things with XML, like

configuration files, is far too high (because of all the other things it is
capable of that you don’t need for such cases) and there was a very real
need for something simpler for that kind of application. Restricting
yourself to a subset of XML doesn’t greatly reduce the cost of writing code
to process it, you still have to use the same APIs. Perhaps it could have
been achieved with some kind of MicroXML, but as a community, we failed to
deliver that.
I agree with Michael here. There was a need, and the world satisfied it.
That’s all.
But in any case, the world could not care less about what WE think at this point.
Both XML and JSON are here to stay. There are good reasons for each one of them.
(1) the two technologies will not be developed completely independently
(users will need to deal with both in the
same time, so that would be a terribly expensive exercise for the industry as a whole) and
(2) JSON people will learn something from what has been developed in the
XML world , and they won’t start from scratch , aka
from were XML started in 1996.
We can dream, can’t we !?? :-)

Well I have no particular beef with the format itself other than the lack
of tools. Now that we have JSONiq I am less bothered about that (assuming
one has the opportunity to use it).

I agree with your ideals (1 and 2 above) too but it should be evident from
the sociology of the JSON community that these things are not going to
happen. You have people putting stuff in JSON databases without thinking
how are we going to get it out and coming up with half-assed solutions for
doing so. This is not progress and this is not good.

[Michael Kay]

I agree with your ideals (1 and 2 above) too but it should be evident from the sociology of the JSON community that these things are not going to happen. You have people putting stuff in JSON databases without thinking how are we going to get it out and coming up with half-assed solutions for doing so. This is not progress and this is not good.

JSON wasn’t designed for databases any more than XML was. Both were designed for messages. XML at least was designed for long-life messages, while JSON was designed for throwaway messages. I think that doing anything more with JSON than reading and writing it is asking for trouble, unless and until it acquires all the layers on top that will remove its attractive simplicity.

Michael Kay
Saxonica
_______________________________________________
***@x-query.com
http://x-query.com/mailman/

[daniela florescu]

. I think that doing anything more with JSON than reading and writing it is asking for trouble,

Huh, Michael, I am not sure I agree with you.

The same way the world expressed the discomfort with the complexity of XML for a LOOOONG time, before,
being unheard, they went ahead and build something to avoid the discomfort… (aka JSON)…...

….the same way the wold today expresses a HUGE need to actually do MORE with JSON then reading and
writing. Aka sophisticated querying and updating.

Ignore that huge need… and they’ll come up with SOME solution to satisfy for their need…..

And as Ihe said, I’m not sure that solution will be pretty to look at …. will be a half baked one.

That seems like burying the head in the sand, pretending not to see what the industry needs.

(and yes, all technologies start simple at the beginning, until they are not anymore …that’s Ok)

Dana
_______________________________________________
***@x-query.co

[Ihe Onwuka]

Post by Ihe Onwuka
I agree with your ideals (1 and 2 above) too but it should be evident

from the sociology of the JSON community that these things are not going to
happen. You have people putting stuff in JSON databases without thinking
how are we going to get it out and coming up with half-assed solutions for
doing so. This is not progress and this is not good.
JSON wasn’t designed for databases any more than XML was. Both were
designed for messages. XML at least was designed for long-life messages,
while JSON was designed for throwaway messages.

I think XML is a bit more useful than that.

It's a natural fit for data scraped from the web and if you buy into the
tools it has a value as a staging format because it is easily convertible.

[daniela florescu]

JSON wasn’t designed for databases any more than XML was. Both were designed for messages. XML at least was designed for long-life messages, while JSON was designed for throwaway messages.

I agree that both were (originally) designed for messages. But I don’t think it has anything to do with the level of persistence
or their lifetime.

Let’s ignore for a second why they were created in the first place, and let’s look at how are they used now — very often
the original intent is not the final use. And, of course, both are used in more circumstances then simple messaging, but
let’s focus on the message use case.

XML is now used as a message format between INDEPENDENT systems (e.g UBS wants to talk to Bank of America).
They have NO CHOICE then to use XML is this case, because:
- they need a standard (if things go wrong
 who’s fault it is !?)
- they need a pre-agreed schema ( otherwise how do they know they understand each other ?)

JSON is of no use in this case, at least for now, because (a) not a standard and (b) doesn’t have a schema

JSON is used today as a message format between client/server parts of the SAME system. There speed is the main requirement,
and XML is too heavy in size and parsing time, so it is of no use.

==========

In both cases, those messages will be logged, and hence persist somewhere.

Nobody throws away data those days, of a variety of reasons
 big data analysis (or at least the hope for it..), audit, security, etc.


So, no, I don’t think XML vs. JSON is a matter of persistence vs. not.


Best regards, have a great weekend
Dana

[Ihe Onwuka]

Post by daniela florescu
In both cases, those messages will be logged, and hence persist somewhere.
Nobody throws away data those days, of a variety of reasons
 big data
analysis (or at least the hope for it..), audit, security, etc.
So, no, I don’t think XML vs. JSON is a matter of persistence vs. not.

But it usually the case that the format in which the data comes at you is
not suitable for the purpose you want to use it.

So whereas you may want to persist it, it doesn't follow that it should be
analyzed in the same form, ergo it doesn't follow that you should/must
build a data manipulation framework on top of that format (in this case we
are talking about JSON).

What you can do instead is transform it into a format that already has the
necessary tools support hence we have usually seen this stuff being sucked
into spreadsheets and relational databases. OK that stops being feasible
with "Big Data" and semi-structured data, but if it takes as long as you
say to build a query language for semi-structured data then clearly that's
not really viable. The question is are the so-called SQL++ hacks good
enough. Personally I think it is madness to take a punt on ropey unproven
software if there is a working alternative.

As for Big Data. The idea that we must have something that sucks everything
up is not IMO broadly applicable it's just people lacking the appropriate
skills to pre-process the data aping Google irrespective that they are not
solving the same type of problem.

[daniela florescu]

Well I have no particular beef with the format itself other than the lack of tools. Now that we have JSONiq I am less bothered about that (assuming one has the opportunity to use it).

Well, JSONiq is only implemented by Zorba (and another implementation in IBM middle tier).

And Zorba is a dead piece of code.

So, having “JSONiq” as a specification
...doesn’t mean much, isn’t it ? Unless is adopted by other XQuery processors.
(which I cross fingers they will do
)

I agree with your ideals (1 and 2 above) too but it should be evident from the sociology of the JSON community that these things are not going to happen.

Well
 nope. Not clear at all.

I started working on query languages for XML in 1996.

Same as now, the whole industry was for using SQL for querying XML — including ME, I had
a system running, a bunch of PhD students working on that, etc.

The decision of the W3C NOT to use SQL for that purpose was taken in 2001.

5 years later. You know how many query languages have been proposed during those 5 years ?
Tons: UnQL, XML-QL, etc, etc.

Those things need TIME.

People need to try SQL first, before they realize it’s a dead end.

MarkLogic needs to try Javascript on the server side, before realizes that’s a dead end.

The industry moves MUCH, MUCH slower that one can expect.

You have people putting stuff in JSON databases without thinking how are we going to get it out and coming up with half-assed solutions for doing so. This is not progress and this is not good.

Again, patience is golden :-)

There will be tons of those half baked solutions (MongoDB’s JSON query language, CouchDB’s..), before people realize that this
is not going anywhere
.some of those databases will be acquired, never to be seen again, them or their query languages
.etc.

===============

Honestly, I give it 5 years for the JSON query languages to stabilize. 2020 would be my estimate. Even later if there is a database bubble crash in the
meantime.

Best
Dana

[Ihe Onwuka]

I wish I could agree with you but I think it is different this time.

Couple of days ago I saw an update on the Scala group, somebody saying that
the upsurge of interest in Spark could be the killer app that catapults
Scala into the mainstream. Much as I would like to see it happen for a
functional programming language, everybody except Scala dev's knows that
the language is just too damn complicated to ever go mainstream.

Even if this criticism could not be levelled at Scala, suspend disbelief
for a minute and accept that Spark is indeed this killer app, alas it is
not going to catapult Scala anywhere because the people employed in that
domain will demand that it is delivered in Python and/or R and the people
that hire them will acquiesce and say verily so.

In the 1990's the bell tolled for the Cobol mainframe programmer. It's not
like that now. On the JVM, the message is not adapt to Scala/Clojure or die
it's don't worry mate stick to what you know and Java 8 will bail us out.

The IT industry has presided over the widepsread and rudimentary
amateurisation of software development. So when the right solution comes
along it encounters a rearguard resistance from people who depend on the
technological status quo for their jobs and who roll out their stock in
trade objections (performance usually high up on the list). It's not like
1990 when mainframe programmers were saying I need to learn Unix/C and an
Rdb and/or 5 years later I need to learn Java and what they thought to be
OOP. Hence 20 years later we are still talking Java, Javascript and SQL and
5 years on they will be looking at Java 10 and still writing Fortran with
classes. The industry goes along with it because they can continue to
source bodies cheaply.

I absolutely agree that what you said should be the way it is goes but I
don't see how it is going to happen with the vested agenda's at play.

Post by Ihe Onwuka
Well I have no particular beef with the format itself other than the lack
of tools. Now that we have JSONiq I am less bothered about that (assuming
one has the opportunity to use it).
Well, JSONiq is only implemented by Zorba (and another implementation in IBM middle tier).
And Zorba is a dead piece of code.
So, having “JSONiq” as a specification
...doesn’t mean much, isn’t it ?
Unless is adopted by other XQuery processors.
(which I cross fingers they will do
)
I agree with your ideals (1 and 2 above) too but it should be evident from
the sociology of the JSON community that these things are not going to
happen.
Well
 nope. Not clear at all.
I started working on query languages for XML in 1996.
Same as now, the whole industry was for using SQL for querying XML —
including ME, I had
a system running, a bunch of PhD students working on that, etc.
The decision of the W3C NOT to use SQL for that purpose was taken in 2001.
5 years later. You know how many query languages have been proposed during those 5 years ?
Tons: UnQL, XML-QL, etc, etc.
Those things need TIME.
People need to try SQL first, before they realize it’s a dead end.
MarkLogic needs to try Javascript on the server side, before realizes
that’s a dead end.
The industry moves MUCH, MUCH slower that one can expect.
You have people putting stuff in JSON databases without thinking how are
we going to get it out and coming up with half-assed solutions for doing
so. This is not progress and this is not good.
Again, patience is golden :-)
There will be tons of those half baked solutions (MongoDB’s JSON query
language, CouchDB’s..), before people realize that this
is not going anywhere
.some of those databases will be acquired, never to
be seen again, them or their query languages
.etc.
===============
Honestly, I give it 5 years for the JSON query languages to stabilize.
2020 would be my estimate. Even later if there is a database bubble crash
in the
meantime.
Best
Dana

[daniela florescu]

Depends what you call “mainstream” (or successful) for a programming language
...

Is it calculated in (1) #developers ?

Or is it calculated in (2) #instances running it ?

Or is it calculated in (3) #millions of dollars revenue from it ?

==========

A. If (2), the Scala will be a HUGE success — because of Spark.

IBM just offered to put more then 3000 (!! and that’s a LARGE number) of developers on Spark.

Cloudera just seemed to have dumped Hadoop to become a Spark reseller.

So
. in terms on number of instances running it, Scala WILL be a huge success.

(how is Tyrpesafe, the company that does the Scala compiler making money 
 that’s another story
)

B. If it is (3) — millions of $$$, I can probably give you the most “cost-effective” programming language in the world.

https://en.wikipedia.org/wiki/K_(programming_language) <https://en.wikipedia.org/wiki/K_(programming_language)>

It’s used for high end financial systems, and a copy of this compiler costs about $1M.

Not bad 
.

Number of developers writing code in this language 
 probably in the 100s
.

C. If it in terms of (1) number of developers 
. are we really sure it matters ?

Who makes money today out of Javascript ? (arguably the most popular programming language 
)

Nobody.

======================

So, which one do you care about ?

I care about (2) and (3).

Best
Dana

Post by Ihe Onwuka
I wish I could agree with you but I think it is different this time.
Couple of days ago I saw an update on the Scala group, somebody saying that the upsurge of interest in Spark could be the killer app that catapults Scala into the mainstream. Much as I would like to see it happen for a functional programming language, everybody except Scala dev's knows that the language is just too damn complicated to ever go mainstream.
Even if this criticism could not be levelled at Scala, suspend disbelief for a minute and accept that Spark is indeed this killer app, alas it is not going to catapult Scala anywhere because the people employed in that domain will demand that it is delivered in Python and/or R and the people that hire them will acquiesce and say verily so.
In the 1990's the bell tolled for the Cobol mainframe programmer. It's not like that now. On the JVM, the message is not adapt to Scala/Clojure or die it's don't worry mate stick to what you know and Java 8 will bail us out.
The IT industry has presided over the widepsread and rudimentary amateurisation of software development. So when the right solution comes along it encounters a rearguard resistance from people who depend on the technological status quo for their jobs and who roll out their stock in trade objections (performance usually high up on the list). It's not like 1990 when mainframe programmers were saying I need to learn Unix/C and an Rdb and/or 5 years later I need to learn Java and what they thought to be OOP. Hence 20 years later we are still talking Java, Javascript and SQL and 5 years on they will be looking at Java 10 and still writing Fortran with classes. The industry goes along with it because they can continue to source bodies cheaply.
I absolutely agree that what you said should be the way it is goes but I don't see how it is going to happen with the vested agenda's at play.

Well I have no particular beef with the format itself other than the lack of tools. Now that we have JSONiq I am less bothered about that (assuming one has the opportunity to use it).

Well, JSONiq is only implemented by Zorba (and another implementation in IBM middle tier).
And Zorba is a dead piece of code.
So, having “JSONiq” as a specification
...doesn’t mean much, isn’t it ? Unless is adopted by other XQuery processors.
(which I cross fingers they will do
)

I agree with your ideals (1 and 2 above) too but it should be evident from the sociology of the JSON community that these things are not going to happen.

Well
 nope. Not clear at all.
I started working on query languages for XML in 1996.
Same as now, the whole industry was for using SQL for querying XML — including ME, I had
a system running, a bunch of PhD students working on that, etc.
The decision of the W3C NOT to use SQL for that purpose was taken in 2001.
5 years later. You know how many query languages have been proposed during those 5 years ?
Tons: UnQL, XML-QL, etc, etc.
Those things need TIME.
People need to try SQL first, before they realize it’s a dead end.
MarkLogic needs to try Javascript on the server side, before realizes that’s a dead end.
The industry moves MUCH, MUCH slower that one can expect.

You have people putting stuff in JSON databases without thinking how are we going to get it out and coming up with half-assed solutions for doing so. This is not progress and this is not good.

Again, patience is golden :-)
There will be tons of those half baked solutions (MongoDB’s JSON query language, CouchDB’s..), before people realize that this
is not going anywhere
.some of those databases will be acquired, never to be seen again, them or their query languages
.etc.
===============
Honestly, I give it 5 years for the JSON query languages to stabilize. 2020 would be my estimate. Even later if there is a database bubble crash in the
meantime.
Best
Dana

[Ihe Onwuka]

Post by daniela florescu
Depends what you call “mainstream” (or successful) for a programming
language
...
Is it calculated in (1) #developers ?
Or is it calculated in (2) #instances running it ?
Or is it calculated in (3) #millions of dollars revenue from it ?
==========
A. If (2), the Scala will be a HUGE success — because of Spark.
IBM just offered to put more then 3000 (!! and that’s a LARGE number) of
developers on Spark.

Yes on Spark. Not Scala. Spark.

Cloudera just seemed to have dumped Hadoop to become a Spark reseller.

Post by daniela florescu
So
. in terms on number of instances running it, Scala WILL be a huge success.

Spark supports Java, Python and Scala there is an open source SparkR.

Now let me remind you of your opening gambit in the discussion.

************************************************************
"Even though, because it’s functional, it will be restricted to be used
only by people with CS degrees, and not by
random Joes and Janes who write web sites. The way it is designed it is
intended to make a population of educated programmers
ETREMELY efficient, and NOT to increase the total number of developers to
hundreds of millions.

When being reproached this fact in the past, my answer was always the same:
building a database application should not be for the uneducated.
It’s like building a 30 story building, you don’t do that without a
architect ad a structural engineer.
E.g. if you want to eradicate a grave neurological disease, you don’t lower
the bar to allow anyone from the street to perform a neurosurgery,
you just make the existing neurosurgents more productive."
*************************************************************

Now you tell me what you think is going to happen.

(how is Tyrpesafe, the company that does the Scala compiler making money 


Post by daniela florescu
that’s another story
)
B. If it is (3) — millions of $$$, I can probably give you the most
“cost-effective” programming language in the world.
https://en.wikipedia.org/wiki/K_(programming_language)
It’s used for high end financial systems, and a copy of this compiler costs about $1M.
Not bad 
.
Number of developers writing code in this language 
 probably in the 100s
.
C. If it in terms of (1) number of developers 
. are we really sure it matters ?

Yes.

Post by daniela florescu
Who makes money today out of Javascript ? (arguably the most popular
programming language 
)
Nobody.

Loads of people make a living out of it.

Post by daniela florescu
======================
So, which one do you care about ?
I care about (2) and (3).

If the answer were 2 XML would be deemed to be a raging success. But of
XQuery's woes you said "So
. I think it is simply a question of 
. there is
no market for XML 

(aka no enough MONEY in the market)."

Similarly having loads of instances running Scala has to translate in some
meaningful way (beyond making money for Databricks and co). Show me the
money.

Post by daniela florescu
I wish I could agree with you but I think it is different this time.
Couple of days ago I saw an update on the Scala group, somebody saying
that the upsurge of interest in Spark could be the killer app that
catapults Scala into the mainstream. Much as I would like to see it happen
for a functional programming language, everybody except Scala dev's knows
that the language is just too damn complicated to ever go mainstream.
Even if this criticism could not be levelled at Scala, suspend disbelief
for a minute and accept that Spark is indeed this killer app, alas it is
not going to catapult Scala anywhere because the people employed in that
domain will demand that it is delivered in Python and/or R and the people
that hire them will acquiesce and say verily so.
In the 1990's the bell tolled for the Cobol mainframe programmer. It's not
like that now. On the JVM, the message is not adapt to Scala/Clojure or die
it's don't worry mate stick to what you know and Java 8 will bail us out.
The IT industry has presided over the widepsread and rudimentary
amateurisation of software development. So when the right solution comes
along it encounters a rearguard resistance from people who depend on the
technological status quo for their jobs and who roll out their stock in
trade objections (performance usually high up on the list). It's not like
1990 when mainframe programmers were saying I need to learn Unix/C and an
Rdb and/or 5 years later I need to learn Java and what they thought to be
OOP. Hence 20 years later we are still talking Java, Javascript and SQL and
5 years on they will be looking at Java 10 and still writing Fortran with
classes. The industry goes along with it because they can continue to
source bodies cheaply.
I absolutely agree that what you said should be the way it is goes but I
don't see how it is going to happen with the vested agenda's at play.

Post by Ihe Onwuka
Well I have no particular beef with the format itself other than the lack
of tools. Now that we have JSONiq I am less bothered about that (assuming
one has the opportunity to use it).
Well, JSONiq is only implemented by Zorba (and another implementation in
IBM middle tier).
And Zorba is a dead piece of code.
So, having “JSONiq” as a specification
...doesn’t mean much, isn’t it ?
Unless is adopted by other XQuery processors.
(which I cross fingers they will do
)
I agree with your ideals (1 and 2 above) too but it should be evident
from the sociology of the JSON community that these things are not going to
happen.
Well
 nope. Not clear at all.
I started working on query languages for XML in 1996.
Same as now, the whole industry was for using SQL for querying XML —
including ME, I had
a system running, a bunch of PhD students working on that, etc.
The decision of the W3C NOT to use SQL for that purpose was taken in 2001.
5 years later. You know how many query languages have been proposed
during those 5 years ?
Tons: UnQL, XML-QL, etc, etc.
Those things need TIME.
People need to try SQL first, before they realize it’s a dead end.
MarkLogic needs to try Javascript on the server side, before realizes
that’s a dead end.
The industry moves MUCH, MUCH slower that one can expect.
You have people putting stuff in JSON databases without thinking how are
we going to get it out and coming up with half-assed solutions for doing
so. This is not progress and this is not good.
Again, patience is golden :-)
There will be tons of those half baked solutions (MongoDB’s JSON query
language, CouchDB’s..), before people realize that this
is not going anywhere
.some of those databases will be acquired, never
to be seen again, them or their query languages
.etc.
===============
Honestly, I give it 5 years for the JSON query languages to stabilize.
2020 would be my estimate. Even later if there is a database bubble crash
in the
meantime.
Best
Dana

[Michael Kay]

"Even though, because it’s functional, it will be restricted to be used only by people with CS degrees, and not by
random Joes and Janes who write web sites.

Many people have reported that to use XSLT successfully, you either have to have a CS degree, or you have to be a non-programmer. The people who struggle are those who want it to be like the only programming language they have used in the past. Functional programming comes much more naturally to non-programmers than to those whose minds have been warped by imperative programming.

Having said that, we do our best in XSLT to hide the technicalities: a non-programmer isn’t going to think of the apply-templates mechanism as a recursive descent with polymorphic function despatch, and they aren’t going to think of xsl:iterate as a fold operation taking a function as its argument.

Michael Kay
Saxonica

[Ihe Onwuka]

Post by Ihe Onwuka
"Even though, because it’s functional, it will be restricted to be used
only by people with CS degrees, and not by
random Joes and Janes who write web sites.
Many people have reported that to use XSLT successfully, you either have
to have a CS degree, or you have to be a non-programmer.

I learnt XSLT before my CS degree, my conceptual understanding hasn't
changed, but back then I would not have been able to grok you the
recursion sometimes needed for things like string handling in XSLT 1.0.

All a degree does in this context is increase the likelihood that you have
been exposed to some sort of functional programming and I believe it is
still possible to get through most degree programmes without encountering
it.

Post by Ihe Onwuka
The people who struggle are those who want it to be like the only
programming language they have used in the past. Functional programming
comes much more naturally to non-programmers than to those whose minds have
been warped by imperative programming.

True. This is the dilemma for Scala (or should be). They always want
people already steeped in the Java ecosystem and that makes sense but the
same people come from the population that will have the most difficulty
adapting to Functional programming and that makes no sense.

Post by Ihe Onwuka
Having said that, we do our best in XSLT to hide the technicalities: a
non-programmer isn’t going to think of the apply-templates mechanism as a
recursive descent with polymorphic function despatch, and they aren’t going
to think of xsl:iterate as a fold operation taking a function as its
argument.

It's how it's presented to the experienced programmer that is the issue. My
metaphor is that an XSLT processor converts an XML document to a stream of
events which you have to write handlers for in the form of template rules.
However that only covers XSLT's push paradigm and although I'd like to say
push is all you need I don't think it's true, sometimes you have to pull.

[Ihe Onwuka]

Post by Ihe Onwuka
It's how it's presented to the experienced programmer that is the issue.

My metaphor is that an XSLT processor converts an XML document to a stream
of events which you have to write handlers for in the form of template
rules. However that only covers XSLT's push paradigm and although I'd like
to say push is all you need I don't think it's true, sometimes you have to
pull.

I should have added that the processor has built in event handlers so you
only need to write handlers for required functionality that is at variance
with the default for that node type.

A non-programmer has no problem accepting this but it is alien to the
experienced programmer for processing that he has not explicitly
controlled to occur and it screws them up because they like to step through
code with debuggers. In fact that is often their prime modus operandi

[Pavel Velikhov]

Well, SQL is a functional language too (if you ignore the various sublanguages for stored procedures). And I have seem many students struggle with it. They are used to assigning values in loops and the whole concept of SQL is alien to them. But in the end all experienced engineers master it without a problem. (Even though the % of people that can use GroupBy/Having, Window, Cube and outerjoins is tiny). At some point the understand that they *have* to learn, and that just does it.

It's how it's presented to the experienced programmer that is the issue. My metaphor is that an XSLT processor converts an XML document to a stream of events which you have to write handlers for in the form of template rules. However that only covers XSLT's push paradigm and although I'd like to say push is all you need I don't think it's true, sometimes you have to pull.
I should have added that the processor has built in event handlers so you only need to write handlers for required functionality that is at variance with the default for that node type.
A non-programmer has no problem accepting this but it is alien to the experienced programmer for processing that he has not explicitly controlled to occur and it screws them up because they like to step through code with debuggers. In fact that is often their prime modus operandi
_______________________________________________
http://x-query.com/mailman/listinfo/talk

С уважеМОеЌ,
Павел ВелОхПв
***@gmail.com

[daniela florescu]

Ihe,

the discussion starts to take too many threads for my brain the be able to wrap around it.
But I’ll answer some of the questions that came up in your emails, one by one.

1. Scala WILL be used. Spark is implemented in Scala. So there will be millions of Scala instances
running.

If ... Typesafe (the company who does the Scala compiler) will make money out of this: I would say no.
I followed their business plan
. and honestly, I am not impressed. I think they have very good engineers but no clue
how to turn this into money. And that’s unfortunate because they COULD make money out of it,

2. About what’s more important for the success of a programming language: #installations, # developers or #millions$$.

Well, it’s subjective.

Honestly, all I care is that (a) there is an important problem that people need to solve and (b) that programming language is the
best tool to solve that problem.

According to this, yes, XSLT and XQuery do the job.

And, also, I don’t think either XSLT or XQuery or JSOniq are the kind of languages that increase the number of developers to millions.

3. The idea that we need “millions of developers” gets me crazy. No, we don’t. We just need smarter people with better tools.
(A good tool like Oxygen has more effect on the success of XQuery then anything we cold have done to design it better
.)

E.g. today there was an article that made the tour of the Internet: "US needs 5 million data scientists by 2017.”

Now contrast that with the little experiment that I was running recently: here is the simplest statistical problem you can imagine.

You have 3 boxes and 6 balls: 3 white and 3 back. You put the balls in the boxes, 2 in each. I take one ball out of one box and it is black.
What is the probability that the secod ball in the same box is also black ?

I asked this question to CTOs of data science companies, Phd in math, Phd In CS, the head of machine learning at Google, hordes of “data scientists”.

Out of tens of people I asked, I got only TWO correct answers: both where Phd in physics. (not the CTOs, not the head of machine learning..)

Now, tell me : if US hires 5 million (!!) data scientists in the next 2 years, what’s the probability that ANY good data science, and some reliable
results come out of that! ?

You know the story: garbage in, garbage out ?

So, no I don’t believe that hiring hordes of uneducated developers helps. (maybe it’s my ex-communist snobbish view concerning the education, or lack thereof..)

Just hire less developers, but smarter ones, and give them very good tools to do their job, that’s all.

4. However, even I don’t believe that we need millions of developers writing XQuery, the question still remains:

Did we do the best we can to market XQuery ? Of course not. Books are scarce, the little material is badly written, there are no classes, etc.

The W3C XQuery web site had “to be added..” for 8 YEARS !!!!! (don’t even get me started 
)

The community is split, and instead of getting together to try to make a common effort to market their common interest, they each one fights for
showing off THEIR muscles, etc.

The XQuery community spent no efforts into marketing, and unfortunately a programming language, like any product, needs MARKETING.

The SQL community spent TONS of efforts for marketing it
 long ago
.and now you can see the results.

Scala community spends TONS of efforts for marketing. Etc. Etc.

Well. I am not sure how to solve that, other then to open my (big) mouth every time I get a chance. (and yes, usually the results is
 "Dana is grumpy" :-)

A more community-based effort would be needed to better explain to the world what XQuery is and what they can do with it.

And yes the fact that MarkLogic pretends that they’ve never heard of XQuery in their life and sells this as “their” technology
to their customers doesn’t help XQuery either. (and honestly, I think it’s dumb business mistake for them too
.but who am I to judge....).



===========



A separate email about the market of XQuery.

Best
Dana

Post by daniela florescu
Depends what you call “mainstream” (or successful) for a programming language
...
Is it calculated in (1) #developers ?
Or is it calculated in (2) #instances running it ?
Or is it calculated in (3) #millions of dollars revenue from it ?
==========
A. If (2), the Scala will be a HUGE success — because of Spark.
IBM just offered to put more then 3000 (!! and that’s a LARGE number) of developers on Spark.
Yes on Spark. Not Scala. Spark.
Cloudera just seemed to have dumped Hadoop to become a Spark reseller.
So
. in terms on number of instances running it, Scala WILL be a huge success.
Spark supports Java, Python and Scala there is an open source SparkR.
Now let me remind you of your opening gambit in the discussion.
************************************************************
"Even though, because it’s functional, it will be restricted to be used only by people with CS degrees, and not by
random Joes and Janes who write web sites. The way it is designed it is intended to make a population of educated programmers
ETREMELY efficient, and NOT to increase the total number of developers to hundreds of millions.
When being reproached this fact in the past, my answer was always the same: building a database application should not be for the uneducated.
It’s like building a 30 story building, you don’t do that without a architect ad a structural engineer.
E.g. if you want to eradicate a grave neurological disease, you don’t lower the bar to allow anyone from the street to perform a neurosurgery,
you just make the existing neurosurgents more productive."
*************************************************************
Now you tell me what you think is going to happen.
(how is Tyrpesafe, the company that does the Scala compiler making money 
 that’s another story
)
B. If it is (3) — millions of $$$, I can probably give you the most “cost-effective” programming language in the world.
https://en.wikipedia.org/wiki/K_(programming_language) <https://en.wikipedia.org/wiki/K_(programming_language)>
It’s used for high end financial systems, and a copy of this compiler costs about $1M.
Not bad 
.
Number of developers writing code in this language 
 probably in the 100s
.
C. If it in terms of (1) number of developers 
. are we really sure it matters ?
Yes.
Who makes money today out of Javascript ? (arguably the most popular programming language 
)
Nobody.
Loads of people make a living out of it.
======================
So, which one do you care about ?
I care about (2) and (3).
If the answer were 2 XML would be deemed to be a raging success. But of XQuery's woes you said "So
. I think it is simply a question of 
. there is no market for XML 

(aka no enough MONEY in the market)."
Similarly having loads of instances running Scala has to translate in some meaningful way (beyond making money for Databricks and co). Show me the money.

Post by Ihe Onwuka
I wish I could agree with you but I think it is different this time.
Couple of days ago I saw an update on the Scala group, somebody saying that the upsurge of interest in Spark could be the killer app that catapults Scala into the mainstream. Much as I would like to see it happen for a functional programming language, everybody except Scala dev's knows that the language is just too damn complicated to ever go mainstream.
Even if this criticism could not be levelled at Scala, suspend disbelief for a minute and accept that Spark is indeed this killer app, alas it is not going to catapult Scala anywhere because the people employed in that domain will demand that it is delivered in Python and/or R and the people that hire them will acquiesce and say verily so.
In the 1990's the bell tolled for the Cobol mainframe programmer. It's not like that now. On the JVM, the message is not adapt to Scala/Clojure or die it's don't worry mate stick to what you know and Java 8 will bail us out.
The IT industry has presided over the widepsread and rudimentary amateurisation of software development. So when the right solution comes along it encounters a rearguard resistance from people who depend on the technological status quo for their jobs and who roll out their stock in trade objections (performance usually high up on the list). It's not like 1990 when mainframe programmers were saying I need to learn Unix/C and an Rdb and/or 5 years later I need to learn Java and what they thought to be OOP. Hence 20 years later we are still talking Java, Javascript and SQL and 5 years on they will be looking at Java 10 and still writing Fortran with classes. The industry goes along with it because they can continue to source bodies cheaply.
I absolutely agree that what you said should be the way it is goes but I don't see how it is going to happen with the vested agenda's at play.

Well I have no particular beef with the format itself other than the lack of tools. Now that we have JSONiq I am less bothered about that (assuming one has the opportunity to use it).

Well, JSONiq is only implemented by Zorba (and another implementation in IBM middle tier).
And Zorba is a dead piece of code.
So, having “JSONiq” as a specification
...doesn’t mean much, isn’t it ? Unless is adopted by other XQuery processors.
(which I cross fingers they will do
)

I agree with your ideals (1 and 2 above) too but it should be evident from the sociology of the JSON community that these things are not going to happen.

Well
 nope. Not clear at all.
I started working on query languages for XML in 1996.
Same as now, the whole industry was for using SQL for querying XML — including ME, I had
a system running, a bunch of PhD students working on that, etc.
The decision of the W3C NOT to use SQL for that purpose was taken in 2001.
5 years later. You know how many query languages have been proposed during those 5 years ?
Tons: UnQL, XML-QL, etc, etc.
Those things need TIME.
People need to try SQL first, before they realize it’s a dead end.
MarkLogic needs to try Javascript on the server side, before realizes that’s a dead end.
The industry moves MUCH, MUCH slower that one can expect.

You have people putting stuff in JSON databases without thinking how are we going to get it out and coming up with half-assed solutions for doing so. This is not progress and this is not good.

Again, patience is golden :-)
There will be tons of those half baked solutions (MongoDB’s JSON query language, CouchDB’s..), before people realize that this
is not going anywhere
.some of those databases will be acquired, never to be seen again, them or their query languages
.etc.
===============
Honestly, I give it 5 years for the JSON query languages to stabilize. 2020 would be my estimate. Even later if there is a database bubble crash in the
meantime.
Best
Dana

[Ihe Onwuka]

Post by daniela florescu
Ihe,
the discussion starts to take too many threads for my brain the be able to wrap around it.
But I’ll answer some of the questions that came up in your emails, one by one.
1. Scala WILL be used. Spark is implemented in Scala. So there will be
millions of Scala instances
running.

Scala is a language for which you do need a CS degree. When I say that I
don't literally mean you need a degree but you need knowledge that you are
less likely to have encountered if you have not done a CS degree.

In general functional programming is something you need to learn at college
or at least from an academic, not from other developers and especially not
from non-academics who have written books because they don't know what to
teach or how to teach it.

So where are these Scala programmers going to come from. Well they need to
come from the universities but I don't think universities will produce them
because universities don't like failing students. Back in the 90's FP was
on the 1st year curriculum at UCL. I asked my tutor why it got removed, he
said too many students were failing it and passing the 1st year
programming course was mandatory to getting to year 2, so they changed it.
Happily they have reverted and now teach Haskell in year one. Programming
Scala will be a bloody hard course I don't think enough students
could/would get through it (stayed tune for another post on this).

Post by daniela florescu
If ... Typesafe (the company who does the Scala compiler) will make money
out of this: I would say no.
I followed their business plan
. and honestly, I am not impressed. I think
they have very good engineers but no clue
how to turn this into money. And that’s unfortunate because they COULD
make money out of it,
#installations, # developers or #millions$$.
Well, it’s subjective.
Honestly, all I care is that (a) there is an important problem that people
need to solve and (b) that programming language is the
best tool to solve that problem.
According to this, yes, XSLT and XQuery do the job.
And, also, I don’t think either XSLT or XQuery or JSOniq are the kind of
languages that increase the number of developers to millions.

Agree.

Post by daniela florescu
3. The idea that we need “millions of developers” gets me crazy. No, we
don’t. We just need smarter people with better tools.
(A good tool like Oxygen has more effect on the success of XQuery then
anything we cold have done to design it better
.)

Agree.

Post by daniela florescu
E.g. today there was an article that made the tour of the Internet: "US
needs 5 million data scientists by 2017.”
here is the simplest statistical problem you can imagine.
You have 3 boxes and 6 balls: 3 white and 3 back. You put the balls in the
boxes, 2 in each. I take one ball out of one box and it is black.
What is the probability that the secod ball in the same box is also black ?
I asked this question to CTOs of data science companies, Phd in math, Phd
In CS, the head of machine learning at Google, hordes of “data scientists”.
Out of tens of people I asked, I got only TWO correct answers: both where
Phd in physics. (not the CTOs, not the head of machine learning..)

Pick a Box A.

Sampling is without replaceent.
Chance of 1st ball being black = 3/6 = 1/2
Chance of 2nd ball being black = 2/5.

Chance of both being blank = 1//2 * 2/5 = 1/5.

BUT there are 2 ways for Box A to have 2 black balls

a). Box B could have zero black and Box C 1 black or
b) Box C could have zero black and Box B 1 white.
So multiply by 2

Se P (Box A has 2 black balls) = 1/5 * 2 = 2/5.

BUT there is only a one in 3 chance of selecting Box A

Final answer = 2/5 * 1/3 = 2/15

I settle for partial credit.

Post by daniela florescu
Now, tell me : if US hires 5 million (!!) data scientists in the next 2
years, what’s the probability that ANY good data science, and some reliable
results come out of that! ?
You know the story: garbage in, garbage out ?
So, no I don’t believe that hiring hordes of uneducated developers helps.
(maybe it’s my ex-communist snobbish view concerning the education, or lack
thereof..)
Just hire less developers, but smarter ones, and give them very good tools
to do their job, that’s all.

Agreed

Post by daniela florescu
Ihe,
the discussion starts to take too many threads for my brain the be able to wrap around it.
But I’ll answer some of the questions that came up in your emails, one by one.
1. Scala WILL be used. Spark is implemented in Scala. So there will be
millions of Scala instances
running.
If ... Typesafe (the company who does the Scala compiler) will make money
out of this: I would say no.
I followed their business plan
. and honestly, I am not impressed. I think
they have very good engineers but no clue
how to turn this into money. And that’s unfortunate because they COULD
make money out of it,
#installations, # developers or #millions$$.
Well, it’s subjective.
Honestly, all I care is that (a) there is an important problem that people
need to solve and (b) that programming language is the
best tool to solve that problem.
According to this, yes, XSLT and XQuery do the job.
And, also, I don’t think either XSLT or XQuery or JSOniq are the kind of
languages that increase the number of developers to millions.
3. The idea that we need “millions of developers” gets me crazy. No, we
don’t. We just need smarter people with better tools.
(A good tool like Oxygen has more effect on the success of XQuery then
anything we cold have done to design it better
.)
E.g. today there was an article that made the tour of the Internet: "US
needs 5 million data scientists by 2017.”
here is the simplest statistical problem you can imagine.
You have 3 boxes and 6 balls: 3 white and 3 back. You put the balls in the
boxes, 2 in each. I take one ball out of one box and it is black.
What is the probability that the secod ball in the same box is also black ?
I asked this question to CTOs of data science companies, Phd in math, Phd
In CS, the head of machine learning at Google, hordes of “data scientists”.
Out of tens of people I asked, I got only TWO correct answers: both where
Phd in physics. (not the CTOs, not the head of machine learning..)
Now, tell me : if US hires 5 million (!!) data scientists in the next 2
years, what’s the probability that ANY good data science, and some reliable
results come out of that! ?
You know the story: garbage in, garbage out ?
So, no I don’t believe that hiring hordes of uneducated developers helps.
(maybe it’s my ex-communist snobbish view concerning the education, or lack
thereof..)
Just hire less developers, but smarter ones, and give them very good tools
to do their job, that’s all.
4. However, even I don’t believe that we need millions of developers
Did we do the best we can to market XQuery ? Of course not. Books are
scarce, the little material is badly written, there are no classes, etc.
The W3C XQuery web site had “to be added..” for 8 YEARS !!!!! (don’t even
get me started 
)
The community is split, and instead of getting together to try to make a
common effort to market their common interest, they each one fights for
showing off THEIR muscles, etc.
The XQuery community spent no efforts into marketing, and unfortunately a
programming language, like any product, needs MARKETING.
The SQL community spent TONS of efforts for marketing it
 long ago
.and
now you can see the results.
Scala community spends TONS of efforts for marketing. Etc. Etc.
Well. I am not sure how to solve that, other then to open my (big) mouth
every time I get a chance. (and yes, usually the results is
 "Dana is
grumpy" :-)
A more community-based effort would be needed to better explain to the
world what XQuery is and what they can do with it.
And yes the fact that MarkLogic pretends that they’ve never heard of
XQuery in their life and sells this as “their” technology
to their customers doesn’t help XQuery either. (and honestly, I think
it’s dumb business mistake for them too
.but who am I to judge....).
===========
A separate email about the market of XQuery.
Best
Dana

Post by daniela florescu
Depends what you call “mainstream” (or successful) for a programming
language
...
Is it calculated in (1) #developers ?
Or is it calculated in (2) #instances running it ?
Or is it calculated in (3) #millions of dollars revenue from it ?
==========
A. If (2), the Scala will be a HUGE success — because of Spark.
IBM just offered to put more then 3000 (!! and that’s a LARGE number) of
developers on Spark.

Yes on Spark. Not Scala. Spark.
Cloudera just seemed to have dumped Hadoop to become a Spark reseller.

Post by daniela florescu
So
. in terms on number of instances running it, Scala WILL be a huge success.

Spark supports Java, Python and Scala there is an open source SparkR.
Now let me remind you of your opening gambit in the discussion.
************************************************************
"Even though, because it’s functional, it will be restricted to be used
only by people with CS degrees, and not by
random Joes and Janes who write web sites. The way it is designed it is
intended to make a population of educated programmers
ETREMELY efficient, and NOT to increase the total number of developers to
hundreds of millions.
When being reproached this fact in the past, my answer was always the
same: building a database application should not be for the uneducated.
It’s like building a 30 story building, you don’t do that without a
architect ad a structural engineer.
E.g. if you want to eradicate a grave neurological disease, you don’t
lower the bar to allow anyone from the street to perform a neurosurgery,
you just make the existing neurosurgents more productive."
*************************************************************
Now you tell me what you think is going to happen.
(how is Tyrpesafe, the company that does the Scala compiler making money 


Post by daniela florescu
that’s another story
)
B. If it is (3) — millions of $$$, I can probably give you the most
“cost-effective” programming language in the world.
https://en.wikipedia.org/wiki/K_(programming_language)
It’s used for high end financial systems, and a copy of this compiler
costs about $1M.
Not bad 
.
Number of developers writing code in this language 
 probably in the 100s
.
C. If it in terms of (1) number of developers 
. are we really sure it matters ?

Yes.

Post by daniela florescu
Who makes money today out of Javascript ? (arguably the most popular
programming language 
)
Nobody.

Loads of people make a living out of it.

Post by daniela florescu
======================
So, which one do you care about ?
I care about (2) and (3).

If the answer were 2 XML would be deemed to be a raging success. But of
XQuery's woes you said "So
. I think it is simply a question of 
. there
is no market for XML 

(aka no enough MONEY in the market)."
Similarly having loads of instances running Scala has to translate in some
meaningful way (beyond making money for Databricks and co). Show me the
money.

Post by daniela florescu
I wish I could agree with you but I think it is different this time.
Couple of days ago I saw an update on the Scala group, somebody saying
that the upsurge of interest in Spark could be the killer app that
catapults Scala into the mainstream. Much as I would like to see it happen
for a functional programming language, everybody except Scala dev's knows
that the language is just too damn complicated to ever go mainstream.
Even if this criticism could not be levelled at Scala, suspend disbelief
for a minute and accept that Spark is indeed this killer app, alas it is
not going to catapult Scala anywhere because the people employed in that
domain will demand that it is delivered in Python and/or R and the people
that hire them will acquiesce and say verily so.
In the 1990's the bell tolled for the Cobol mainframe programmer. It's
not like that now. On the JVM, the message is not adapt to Scala/Clojure or
die it's don't worry mate stick to what you know and Java 8 will bail us
out.
The IT industry has presided over the widepsread and rudimentary
amateurisation of software development. So when the right solution comes
along it encounters a rearguard resistance from people who depend on the
technological status quo for their jobs and who roll out their stock in
trade objections (performance usually high up on the list). It's not like
1990 when mainframe programmers were saying I need to learn Unix/C and an
Rdb and/or 5 years later I need to learn Java and what they thought to be
OOP. Hence 20 years later we are still talking Java, Javascript and SQL and
5 years on they will be looking at Java 10 and still writing Fortran with
classes. The industry goes along with it because they can continue to
source bodies cheaply.
I absolutely agree that what you said should be the way it is goes but I
don't see how it is going to happen with the vested agenda's at play.

Post by Ihe Onwuka
Well I have no particular beef with the format itself other than the
lack of tools. Now that we have JSONiq I am less bothered about that
(assuming one has the opportunity to use it).
Well, JSONiq is only implemented by Zorba (and another implementation in
IBM middle tier).
And Zorba is a dead piece of code.
So, having “JSONiq” as a specification
...doesn’t mean much, isn’t it ?
Unless is adopted by other XQuery processors.
(which I cross fingers they will do
)
I agree with your ideals (1 and 2 above) too but it should be evident
from the sociology of the JSON community that these things are not going to
happen.
Well
 nope. Not clear at all.
I started working on query languages for XML in 1996.
Same as now, the whole industry was for using SQL for querying XML —
including ME, I had
a system running, a bunch of PhD students working on that, etc.
The decision of the W3C NOT to use SQL for that purpose was taken in 2001.
5 years later. You know how many query languages have been proposed
during those 5 years ?
Tons: UnQL, XML-QL, etc, etc.
Those things need TIME.
People need to try SQL first, before they realize it’s a dead end.
MarkLogic needs to try Javascript on the server side, before realizes
that’s a dead end.
The industry moves MUCH, MUCH slower that one can expect.
You have people putting stuff in JSON databases without thinking how are
we going to get it out and coming up with half-assed solutions for doing
so. This is not progress and this is not good.
Again, patience is golden :-)
There will be tons of those half baked solutions (MongoDB’s JSON query
language, CouchDB’s..), before people realize that this
is not going anywhere
.some of those databases will be acquired, never
to be seen again, them or their query languages
.etc.
===============
Honestly, I give it 5 years for the JSON query languages to stabilize.
2020 would be my estimate. Even later if there is a database bubble crash
in the
meantime.
Best
Dana

[Ihe Onwuka]

Post by Ihe Onwuka

Post by Ihe Onwuka
Pick a Box A.

Sampling is without replaceent.
Chance of 1st ball being black = 3/6 = 1/2
Chance of 2nd ball being black = 2/5.
Chance of both being blank = 1//2 * 2/5 = 1/5.
BUT there are 2 ways for Box A to have 2 black balls
a). Box B could have zero black and Box C 1 black or
b) Box C could have zero black and Box B 1 white.
So multiply by 2
Se P (Box A has 2 black balls) = 1/5 * 2 = 2/5.
BUT there is only a one in 3 chance of selecting Box A
Final answer = 2/5 * 1/3 = 2/15
I settle for partial credit.

Good thing I only aimed for partial credit as I got it wrong and I guess it
is because I over-elaborated.

If you've seen 1 black ball there are 5 other possible balls to choose
from only 2 of which are black.

No need to multiply the probability of picking that box (1/3) because that
is given information.

So by sampling without replacement the probability is 2/5.

Final answer.

[Ihe Onwuka]

Post by daniela florescu
Now, tell me : if US hires 5 million (!!) data scientists in the next 2
years, what’s the probability that ANY good data science, and some reliable
results come out of that! ?
You know the story: garbage in, garbage out ?
So, no I don’t believe that hiring hordes of uneducated developers helps.
(maybe it’s my ex-communist snobbish view concerning the education, or lack
thereof..)

This came into my feed this morning and I thought ere we go, another piece
of clickbait but I still couldn't resist. Actually a very very good
article.

See myth 1.

http://www.infoworld.com/article/2936947/big-data/debunked-9-big-data-and-hadoop-myths.html

[Ihe Onwuka]

I still think you are wildly over-estimating this.

Here is the Purdue course. XQuery|Path gets 1 slide in 1 lecture

https://www.cs.purdue.edu/homes/ake/cs348/XMLLec1.ppt

At UCL the only time I recall XML being mentioned was in the context of
setting up an Ant files in the 2nd year Programming course.

Databases in general only merited 1/3 of a course
http://www.cs.ucl.ac.uk/teaching_learning/syllabus/undergrad/2008_logic_and_database_theory/

There is a 3rd year Databases and MIS course which may well offer more the
type of coverage you are suggesting but that is an elective.

There is also this https://www.ucl.ac.uk/dis/taught/pg/INSTG037 that does
the full XML monty, but that is a PG diploma course and it is not run out
of CS.

Ironically about 10 years ago UCL was the source of alot of XML related
research especially for the financial markets.


NB It is also a big part of the BCS Advanced Databases curriculum but again
that course is not mandatory.

I'm not sure about that. I know Harvard does but that is part of a web programming course.
http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-830-database-systems-fall-2010/readings/
http://ocw.mit.edu/courses/civil-and-environmental-engineering/1-264j-database-internet-and-systems-integration-technologies-fall-2013/lecture-notes-exercises/
Look inside the slides of the database classes
 you’ll see XQuery almost
everywhere (at the end of the class
).
Especially that Stanford does it, and most database professors copy the
slides from Stanford.
https://lagunita.stanford.edu/courses/DB/XPath/SelfPaced/courseware/ch-querying_xml/seq-vid-xquery_introduction/
(not that the Stanford professors understand much from XML and XQuery 

but that’s a whole new discussion
)
MIT doesn't, neither does either of the universities I went to.
That’s because Stonebreaker is there, and he wouldn’t be caught dead
teaching XML. I’ll see if I can make him change his mind
about JSON
. :-)
The fact that sir TBL is in MIT too doesn’t help at all either, given the
fact that he was “convinced” by a bunch of good willing people that XML is
evil. (and the semantic web will cure the hunger of the world and the
cancer for sure
)
What are they. Alot of the ones I have read aren't true and seem to be
based on a lack of knowledge about XML
I think I KNOW something about XML — not that I enjoyed learning it, but I HAD to :-)
Want a quick list ? processing instructions, weirdo parsing rules that are
inherited from the 1950’s SGML, weirdo design
of namespaces, XML Schema anyone !? nillable anyone !? Some early design
of Xpath 1.0 (no reserved keywords, semantics of =)
I’ll stop here.
Only those and that would stop me (as a database person) right there to
use XML as a format for data.
Y'see I reckon that if JSON was deployed in many of the domains where XML
is, it too would be hated.
Yes, but less.
People would STILL have to face the fact that they cannot process JSON
with their beloved SQL
. but at least they wouldn’t
add insult to injury by adding the weird things I listed above.
The bitter pill would be somehow easier to swallow.
The database people accept with open arms JSOn right now because they
don’t REALLY understand what JSON is.
Most of them still look at JSON as a nicer syntax then CSV
..they think
it’s flat, or a “little” nested.
The shock will hit them later on :-))))
Best regards
Dana

[Jason Hunter]

DatasTax after 3 yearsof existence is at more then 300M revenue (and less investment from VCs).

I was curious about this figure. Did DataStax really make it close to $300m in revenue?

http://www.datanami.com/2014/09/04/datastax-nabs-106-million-sets-sites-international-expansion/
--
During that period, it more than doubled the number of employees to more than 350, and it increased its annual revenue run rate by more than 125 percent. “The valuation of the company is over $830 million, which is more than double the valuation of the company 12 months ago,” Pfeil said. The privately held company doesn’t share revenue figures.
--

If the company's valuation is $830m and it grew by 125% in the year before, then we can deduce revenue is nowhere close to $300m/yr. It wouldn't even be above $100m/yr.

They raised $190m in VC so far.

-jh-


_______________________________________________
***@x-query.com
http://x-query.com/mailman/listinfo/talk

[daniela florescu]

Jason,

I have their financial analysis slides, but honestly I am not sure if I am supposed to make them public.
(I guess you just have to wait until THEY make them public).

About the valuation, I have no idea.

Not everybody likes to have a 10X valuation, which is clearly a sign of tech bubble over-evaluation.
(and actually such over-evaluations can be very harmful to companies later on if things don’t turn up as expected)

But I do not know exactly the story with their latest evaluation.

Best
Dana

Post by Jason Hunter

DatasTax after 3 yearsof existence is at more then 300M revenue (and less investment from VCs).

I was curious about this figure. Did DataStax really make it close to $300m in revenue?
http://www.datanami.com/2014/09/04/datastax-nabs-106-million-sets-sites-international-expansion/
--
During that period, it more than doubled the number of employees to more than 350, and it increased its annual revenue run rate by more than 125 percent. “The valuation of the company is over $830 million, which is more than double the valuation of the company 12 months ago,” Pfeil said. The privately held company doesn’t share revenue figures.
--
If the company's valuation is $830m and it grew by 125% in the year before, then we can deduce revenue is nowhere close to $300m/yr. It wouldn't even be above $100m/yr.
They raised $190m in VC so far.
-jh-
_______________________________________________
http://x-query.com/mailman/listinfo/talk

_______________________________________________
***@x-query.com
http://x-query.com/mailman/listinfo/talk

[daniela florescu]

Also, Jason,

I think those slides are a little old (things change very fast those days), but still instructive
about the NoSQL market and how it evolves.

https://speakerdeck.com/abchaudhri/considerations-for-using-nosql-technology-on-your-next-it-project-1 <https://speakerdeck.com/abchaudhri/considerations-for-using-nosql-technology-on-your-next-it-project-1>

Enjoy
Dana

Post by daniela florescu
Jason,
I have their financial analysis slides, but honestly I am not sure if I am supposed to make them public.
(I guess you just have to wait until THEY make them public).
About the valuation, I have no idea.
Not everybody likes to have a 10X valuation, which is clearly a sign of tech bubble over-evaluation.
(and actually such over-evaluations can be very harmful to companies later on if things don’t turn up as expected)
But I do not know exactly the story with their latest evaluation.
Best
Dana

Post by Jason Hunter

DatasTax after 3 yearsof existence is at more then 300M revenue (and less investment from VCs).

I was curious about this figure. Did DataStax really make it close to $300m in revenue?
http://www.datanami.com/2014/09/04/datastax-nabs-106-million-sets-sites-international-expansion/
--
During that period, it more than doubled the number of employees to more than 350, and it increased its annual revenue run rate by more than 125 percent. “The valuation of the company is over $830 million, which is more than double the valuation of the company 12 months ago,” Pfeil said. The privately held company doesn’t share revenue figures.
--
If the company's valuation is $830m and it grew by 125% in the year before, then we can deduce revenue is nowhere close to $300m/yr. It wouldn't even be above $100m/yr.
They raised $190m in VC so far.
-jh-
_______________________________________________
http://x-query.com/mailman/listinfo/talk

[daniela florescu]

And another paper about the business of VCs, evaluations, and the damn tech database bubble we are in right now….
which unfortunately has effects on what kind of technology will be “successful” …...

Interesting read, anyway.
http://www.saastr.com/the-dry-bubble-we-may-be-in-what-that-means/#.VYjQ2pBJwQg.facebook <http://www.saastr.com/the-dry-bubble-we-may-be-in-what-that-means/#.VYjQ2pBJwQg.facebook>


Best
Dana

Post by Jason Hunter

DatasTax after 3 yearsof existence is at more then 300M revenue (and less investment from VCs).

I was curious about this figure. Did DataStax really make it close to $300m in revenue?
http://www.datanami.com/2014/09/04/datastax-nabs-106-million-sets-sites-international-expansion/
--
During that period, it more than doubled the number of employees to more than 350, and it increased its annual revenue run rate by more than 125 percent. “The valuation of the company is over $830 million, which is more than double the valuation of the company 12 months ago,” Pfeil said. The privately held company doesn’t share revenue figures.
--
If the company's valuation is $830m and it grew by 125% in the year before, then we can deduce revenue is nowhere close to $300m/yr. It wouldn't even be above $100m/yr.
They raised $190m in VC so far.
-jh-
_______________________________________________
http://x-query.com/mailman/listinfo/talk

[John Snelson]

On 16/06/15 08:21, Ihe Onwuka wrote:
Probably the wrong place to ask but what are the prime factors behind the resistance to adopt XQuery or it's derivatives thereof as a technology for querying and manipulating semi-structured data.

What can you say about why any programming language becomes popular? There are too many counter examples to believe it's because of technical merit, or that they solve problems better or that others can't.

The only answer is that programming is subject to fashions and trends beyond logical understanding. It's an unsatisfactory answer, but it's the only one.

John


--
John Snelson, Lead Engineer http://twitter.com/jpcs
MarkLogic Corporation http://www.marklogic.com

[Michael Kay]

Post by John Snelson
The only answer is that programming is subject to fashions and trends beyond logical understanding. It's an unsatisfactory answer, but it's the only one.

Indeed, some of the most successful languages like Java and PHP succeeded in ways that their original designers never intended.

Michael Kay
Saxonica




_______________________________________________
***@x-query.com
http://x-query.com/mailman/listinfo/talk

[James Fuller]

It may seem like fashion, but using a programming language comes with a
significant investment before becoming useful in said language ... in that
respect I believe programming language adoption is similar to spoken
language in that one learns from their direct and closest community.

So initial exposure to programming whether it be at home, in academia or
work probably is the strongest determining factor for language adoption by
the individual. These choices maybe by fiat (teacher chooses to teach it,
work enforces it, etc) but lets agree that learning a programming language
is significantly harder then putting on any of these outfits.

Loading Image...

So yes, if you are introduced to PHP, lisp, c, python at your first job
then its likely you will learn one of them before learning Objective C.

Learning a spoken/written language is also harder then what most people
imagine ... akin to learning a musical instrument. Learning is punctuated
by local minima of understanding ... a few weeks later breakthroughs are
made. I think whatever gets you through these plateau's is also a strong
adoption factor ... its easier to stop at every stage, especially if you
already know a useful language and can get 'work done'.

Adoption may look like fashion (especially with all the recycling going
on), but its a lot more complicated then that.

J

Post by John Snelson

Post by John Snelson
The only answer is that programming is subject to fashions and trends

beyond logical understanding. It's an unsatisfactory answer, but it's the
only one.
Indeed, some of the most successful languages like Java and PHP succeeded
in ways that their original designers never intended.
Michael Kay
Saxonica
_______________________________________________
http://x-query.com/mailman/listinfo/talk