I can't offer much in the way of advice for Dialect development, but anecdotally it took me maybe 3-4 weeks of reading the Calcite source code every day after work (mostly the tests) and whatever tutorial material I could find to start to wrap my head around the general API and program flow.
(I would consider myself much less talented as a developer than most on the mailing list, though. And with your background in DB's I'd wager it'd be a fair bit easier/quicker for you.) One thing that helps me tremendously these days is Github Copilot and ChatGPT or similar. With Copilot you can start to write out some code that uses Calcite and it'll usually have a pretty good idea of what you're trying to do and offer to complete the rest of the code for you (it does this well in both Java and Kotlin IME). Or you can ask ChatGPT how to implement certain things, like: > "How can I create a new SQL Dialect in Apache Calcite? What classes/interfaces > do I need to extend or implement?" [0] Another thing that helped was to use the new Github Codesearch looking for specific API method usage in public projects. For instance, searching something like `JdbcCatalogSchema` gives 82 results and on page 3 you can learn how to use it from forked Drill repositories. [1] This was the only way I was able to figure out how to construct an `AvaticaJsonHandler` using a local `CalciteConnection` [2] for example. For whatever you can't figure out, I don't think I've seen a question on the mailing list (within reason) go unanswered, the Calcite folks are super friendly. Anyways, I hope some of this advice/anecdata is useful, and good luck! [0]: https://chat.openai.com/share/3d44a572-90d1-4446-bf89-16540f7f929a [1]: https://github.com/search?q=jdbccatalogschema&type=code&p=3 [2]: https://gist.github.com/GavinRay97/080386aa7c0d43d90f1ec1af2bf71571#file-example-kt-L14-L16 On Tue, Jul 11, 2023 at 1:31 PM Julian Hyde <[email protected]> wrote: > I see those problems too. > > A major problem is that when you start a new dialect, there are no tests. > And that’s because our dialect test does not execute queries, only compares > generated SQL (and those SQL fragments need to be entered by hand). There’s > a good reason that our dialect test does not execute queries: connections > to external databases are expensive (you need a license, and cloud > resources) and flaky (valid tests will fail from time to time). > > The moonshot to address this is > https://issues.apache.org/jira/browse/CALCITE-5529. > > I have reached the point where I can execute all queries in > RelToSqlConverterTest against any connection you provide (and I plan to run > them against a reference connection - say Postgres - so that I know that > the results from your connection are valid). The problem is that I don’t > have a means to initialize a connection (create tables and populate data) > and that there are a huge number of errors. > > I would appreciate help working on this. It’s a lot of work, but it will > improve the quality of our dialect system. > > Julian > > > > On Jul 11, 2023, at 6:54 AM, Joeri van Ruth > <[email protected]> wrote: > > > > Hi all, > > > > MonetDB is an open source columnar main-memory SQL database that has > > been around for quite a while. I am a MonetDB developer and I would > > like to add support for MonetDB's SQL dialect to Calcite and > > contribute it to the Calcite project. > > > > Looking at some of the existing Dialect classes I don't expect the > > end result to be a lot of code but I'm a bit concerned that the > > process of getting there will be an endless treadmill of users > > finding yet another deviation and me fixing it. > > > > I'm hoping to ask you here for some guidance on how to develop a new > > dialect in a more systematic way and how to ensure it will not > > immediately fail when someone other than the developer tries to use > > it. > > > > For example, I know you have an extensive test suite, will I be able > > to use that to determine if my implementation is ready, and to > > pinpoint the things I still need to do? Are there particular parts > > of the rest of the code base I should first try to familiarize myself > > with to make the implementation go more smoothly? > > > > Also, what would you expect to be a reasonable time frame for this > > project considering I have good knowledge fo MonetDB but I'm new to > > Calcite? A week? A month? > > > > Any tips and pointers would be appreciated. > > > > Joeri > >
