TeXtalk: an interview with Patrick Gundlach

Welcome to the TeXtalk! We have a very special guest for today: our friend Patrick Gundlach, one of the most active members of TeX.sx, close to 9k+ rep, 92+ badges (including the LuaTeX badge), 116+ answers, one of our representatives at TUG, and LuaTeX expert. Get ready for this great interview!

Paulo Cereda: Could you tell us a bit about you?

Patrick Gundlach: I am 38 years, live in Berlin, Germany and have been a TeX user since 1996. Three years ago I have started my own company based on TeX. We specialize in printed product catalogs.

I am now a board member of DANTE, the German speaking User Group.

Alan Munn: You’re mainly known around here as our resident LuaTeX expert. Are the specific needs of your business use of LaTeX what has sent you in that direction?

Patrick: The product (which is open source, by the way) is based on LuaTeX. The history is the following: I studied computer science, so I have a background and a wish to “automate everything I can”. Every repeated task on a computer is replaced by a makefile, a script or something else.

Paulo: I can relate to that. :)

Patrick: And I love typography from early days. And I had to do some automated typesetting tasks in the past years, mainly bibliographies and lists. Now I was thinking “can good typesetting and automation be joined without losing quality”?

My main objectives are (as written above) product catalogs but also complex data sheets and other database driven documents.

It was obvious that my choice for an automated database publishing software is TeX, but I never liked to program in TeX. It is so awkward for me. All these expandafters, afterassignments and csname .. endcsnames. They really confuse me, even I really like to learn new programming languages.

Then there was LuaTeX.

I almost fell in love with it, because I was able to program in a decent language (Lua) but still can use TeX.

Joseph Wright: So you did not start off with a pure TeX based workflow? (Someone in UK-TUG was involved in the past in catalogue production using pure TeX)

Patrick: I tried several times, but there are a few drawbacks: you have to be really careful about escaping characters (even the “normal ones” like % & and such).

Joseph: Certainly can be tricky (although e-TeX does help).

Patrick: Then, in my experience, every application has to be written from scratch again. So the background is something like “I want to use TeX, but not the programming language of it”.

Joseph: DANTE is a big organisation, certainly compared with UK-TUG. Is there a lot to do as a board member, and do you feel that DANTE has a particular role to play within the wider TeX world (beyond German speakers alone)?

Patrick: Actually, I don’t think that DANTE has a particular role outside of Germany/Austria/Swiss. I have been elected last thursday as a “Beisitzer”, that is a person who can be asked for advice. So I don’t have any idea if there is a lot of work (I hope so).

My aim is to make TeX sexy again.

I think it is an awesome piece of software that ist still up to date, but when I look at the age of members of DANTE (in terms of membership years) there are no youngsters.

Paulo: Speaking of TeX itself, how was your first contact back in 1996?

Patrick: I have read an article about TeX in the German computer magazine “c’t” two years before that and in 1996 I came to university and found a Linux distribution and a notice on the back that it included TeX. So I bought that CD and installed Linux (and TeX) on my system. Which was awful, because I had to learn Linux and TeX at the same time and didn’t know the difference between LaTeX and TeX. A few books from the library helped me.

egreg: Which books, if you can remember?

Patrick: Some German books. It was not the famous Kopka. I remember that I had a script from the “Fern Universität Hagen” (the biggest distance learning university in Germany). I don’t remember any title anymore. I think I also learned from Lamport’s book.

But I remember that I looked into a book about TeX and LaTeX just gave me errors (missing \documentclass and such). I was soooo confused by that. I really hated it, but I wanted to be able to reproduce the nice examples from these books.

Alan: What do you see as the relationship (if any) between LaTeX3 and LuaTeX? (if this is better a question for Joseph, we can leave it out of the discussion entirely.)

Patrick: Too bad IMO that LaTeX3 predates LuaTeX. Although I can perfectly understand the L3 team to “ignore” LuaTeX, I think we have much more power to bring the TeX community forward with LuaTeX. But that would mean to leave LaTeX behind.

Alan: How so? If you were to design the “new” TeX with LuaTeX what would it look like? (I’m assuming backward compatibility wouldn’t be a criterion.)

Patrick: I am not sure. Sometimes LaTeX is unstructured, for example you can open a \subsection{} directly after a \chapter{} and don’t get an error message. So it should have a much more formal input language with validators, but still the power to describe the things you need. I have not decided for me what would be the best, but I guess something like a CSS based markdown format with the ability to add custom macros.

LaTeX is too difficult for the beginner to use, because it mixes programming and input.

I know that there are drawbacks to every approach, but it can’t be that LaTeX (whatever version) is the ultimate description language.

Joseph: I’d say that in that sense there is not so much between anyone involved in “beyond LaTeX2e” thinking.

Certainly, from my point of view on the LaTeX3 team a better “user” experience is very important.

Patrick: Of course, this has nothing to do anymore with LaTeX2. But we can use LuaTeX for that task easily.

Joseph: Where do you feel that XML fits into this area?

Patrick: XML has the great advantage of being the lingua franca in the “data world”. It is standardized and can be validated. It can be hard for a human to type, but once you shift to databases, an XML based workflow is so much easier to handle than anything else. So if we want TeX to be able to process data, we need to make it XML aware.

A great job has been done by David Carlisle with his xmltex (I don’t know passive TeX, sorry), which is a widely used way to get XML into TeX.

With LuaTeX you have the ability to use a validating XML parser directly within your TeX binary (well, if you include one, of course, but that is what I do).

Joseph: So good for database-driven input but less good for “user” work?

Patrick: Exactly. I don’t want to write a letter with <address>Mr. Smith</address> and so on

Marco Daniel: Do you think the implementation of Lua into LaTeX is more user friendly? My point of view is you have to learn two languages.

Patrick: I see your point. If you use LaTeX, I don’t see much reason to use Lua as well. But once you shift to the Lua side, you don’t need LaTeX ;-) It all depends on the type of document you want to typeset. If you want your paper in mathematics / physics to be typeset, then LaTeX is fine. But as soon as you want less (letters for example) or more (complex data sheets), LaTeX is not the best input.

Alan: I find this idea very interesting although hard to wrap my head around. So in principle you want a document markup model that would be entirely Lua based, with TeX completely behind the scenes and more of a black box to the user?

Patrick: Right. TeX could read RTF files, Markdown or Textile input or HTML files. Then LuaTeX would convert it (with some CSS files) to PDF. You still get the finest typography, but easy input. For complex things, you still can switch to LaTeX input.

Marco: Are you planning to write some short articles about the drawbacks and of course benefits of LuaLaTeX for the DTK or TUG?

Patrick: I am writing LuaTeX introductory material for the TUGboat and the DTK (German TeX magazine).

And I am still writing on my book, but it takes me much more time than I which it does. So I am still not finished.

I will present my ideas in October on the EuroTeX conference in the Netherlands and then I have some english papers that could be published in the TUGboat.

Marco: I think you are one of the pioneers of LuaLaTeX and I hope you know this ;-)

Patrick: While I am certainly one of the pioneers of the Lua part of LuaTeX (a few things like tex.linebreak() have been implemented on my request), there are quite a few people that were more active with LuaLaTeX than I was: for example Manuel Pégourié-Gonnard, Élie Roux and Khaled Hosney.

Paulo: Recently you published a very nice package, lua-visual-debug. Could you tell us a bit about it? :)

Patrick: There are some things that I do and really like at the same time. The package is based on a question here on TeX.sx: How can I visualize boxes?

I have seen something like that a few years ago from Hans Hagen and I thought that LuaTeX should be able to do that. After a few attempts I found out about the atbegshi package from the awesome Heiko Oberdiek and then I sat down and wrote the package. It was so much fun, because the code is so simple (it is in effect just a loop that inserts a few PDF instructions depending on the current item in the TeX output) but the output is impressive.

Actually it is my first CTAN submission that has has reached a user base > 1.

Almost all Lua code I write here follows the same rule. Look at the nodes that TeX generate and insert a few new nodes here and there. Nodes is the basic data structure TeX generates just before writing the PDF.

My answer to One letter word at the end of line is using pretty much the same algorithm. So there is no black magic involved, just a simple while-loop. Once you know this, you can do nice stuff. For example the answer to the question.

And there is another problem: LuaLaTeX depends on luaotfload (when you use fontspec), which is a beast. I really mean: a beast. It is big and complex and nobody understands it.

Alan: I also liked your “show hyphenation marks” code that I used in this answer: babel: english, american, USenglish

Patrick: Which is the same while-loop as in all the other answers. :)

David Carlisle: in a recent Lua/LaTeX3 thread on the main site, the subject of stability came up. Do you think LuaTeX is stable enough to commit a major format like LaTeX to it (or for a major publisher like AMS)? As you know there have been several new TeX engines and if LaTeX had switched from TeX to eTeX to NTS to Omega to XeTeX to LuaTeX, it’s not clear that there would have ever been a stable platform for package development. but presumably at some point one has to jump.

Patrick: Good question. There are sometimes bugs that are really awful. There was one version that eliminated almost all hyphenation.

Joseph: Oops :-)

Patrick: I would never ever trust a moving target that I don’t understand.

Joseph: The implication is therefore “not yet”.

Patrick: Right. I have a different situation because I use only the core part of LuaTeX and write everything myself, even the font loader. So as long as I don’t update LuaTeX, I have a reliable workflow.

Joseph: I assume that for your work you maintain a system of “known” status, so bugs are not an issue in the sense that they are “stable”.

Patrick: Bugs like the mentioned one (with the hyphenation) don’t really hit me, because then I just go back to the previous version of LuaTeX. But if for example luaotfload depends on a newer version, you don’t have that choice. Currently I don’t mind such things and I do use LuaLaTeX.

Alan: But really crucial, especially if LuaLaTeX is going to successfully replace XeLaTeX (which is essential these days for anyone who is using non-Latin scripts, RTL etc.).

Joseph: The font loading situation, certainly for fontspec, is on my “worry list”. 🙂

Patrick: The weak part in my opinion is luaotfload. It is a great piece of software (it can do so much), but it is unmaintainable. It is based on the font loader of ConTeXt. So bugs can only get fixed on the context side, but if Hans does not consider something as a bug (because he has a fix somewhere else in the code) we have a big problem.

There are numerous places where we already hit luaotfload bugs (for example there is no way to load type1 fonts with luaotfload). So one day one active group need to replace that beast with something less capable, but stable and easier to maintain.

Joseph: Perhaps we should start some serious discussion about this.

Patrick: Yes, that woud be good.

Paulo: How did you become aware of the TeX.sx community?

Patrick: I can’t remember, it is more than a year ago :) I’ve probably found it “on Google” or read about it on Hacker News (news.ycombinator.com).

egreg: Apart from the articles you already mentioned, what are your next (Lua)TeX related projects?

Patrick: my open source software must be better (more features and English documentation for example) and I really would like to implement a different font loader for LaTeX. But that would take too much time.

I will try to push LuaTeX in the TeX community to make it more attractive to new TeX users. They can then write the fontloader instead. ;)

But it actually depends on interesting questions here on TeX.sx.

I have also thought about a typesetting framework such as Rails is for the web.

You just need to fill out the blanks and you get a typesetting environment that can be used to produce markdown based PDFs.

Paulo: I’ll post a question: “How to implement a different font loader using LuaTeX?” :P

Patrick: You’d get a minimal answer then :)

egreg: You’ve recently reached 10000 rep (a very big step); what do you feel about the community?

Patrick: Oh, no, I am still at 8k, so I am just a small light.

egreg: Oh, sorry, I looked at the global rep. :( So Paulo doesn’t still need to hand you his credit card number. :) But the main question remains. :)

Patrick: The community here is really awesome. Really, I mean it. I have never spent so much time answering TeX question and talking to other TeX people (well, there was a time I had to write my diploma thesis where I have done a lot for the ConTeXt community, the contextgarden.net web server and related things).

It is the best thing that happened to the TeX community for getting support. It would be nice if it could handle other languages as well, but then the community is probably to small.

TeX is hard to learn and to use, so a good community is very important. And we have been able to attract the experts and the “glue people” that keep the experts here.

I ask a lot of questions myself and I know that I can get real “professional” help here. And I search this site a lot. Most of my questions are covered here.

BTW: I ignore the TikZ questions here, so I only see half of the TeX.sx questions. This makes it much easier to follow TeX.sx.

Alan: What is a “glue” person? Someone who asks interesting enough questions for experts to be interested in answering?

Patrick: A glue person could be: a moderator who gives a friendly welcome to new members, people like Paulo who do interviews and hang around in chat to help people and might not have 60k rep, but still know a lot.

So a person between the “pure answer” and the person who posts a question.

It might be even a person who edits posts and fixes spelling mistakes and bad grammar. I know that you did this in some of my posts and I am really thankful for that.

This is one of the things I like here. It is not just about answers, but making the site a better experience.

Alan: I’m happy to do that, especially with good answers. But I know what you mean. I also like the fact that most of us will help each other to improve answers via comments rather than just posting competing answers. This promotes a much better environment overall.

Patrick: I have unsubscribed to all mailing lists. For example the German list tex-d-l has such a bad signal-to-noise ratio: there is about one mail per day and every few days a thread starts in the sense: “don’t post HTML here” with lots of answers. So for every question asked, there are three unnecessary mails.

While I love to chat, I only chat where it is welcome by others (I have unsubscribed to all TeX-related mailing lists).

Alan: Yes, I agree with this. I’ve all but abandoned the Mac TeX list, and now only subscribe to a couple of developer heavy lists which have relatively low noise.

Patrick: I am still on the LuaTeX list, of course.

Paulo: How do you feel being portraited in the following awesome image:

Patrick: I am the second right on the top row. It is really an honor to be on that picture.

Paulo: You have an epic gardener hat in that image. :D

Patrick: Coming back to the community: What I like about the community is that it feels that everyone is so friendly. There are almost no RTFM answers. This is one thing I really don’t like in the groups/lists that I know. Unless it is a clear pointer to a manual. But when I ask, it is because the manual is unclear to me.

Alan: We are a remarkably restrained bunch. :-) But I think the fact that it’s quite easy to find duplicates and then point people to existing answers is a big reason for this. That’s the real problem with newsgroups: they’re lousy as repositories of old information. And slogging though a google groups search isn’t very nice either.

Patrick: Thats a good explanation.

Paulo: Sorry if this is a newbie question: do you use texlua or lua? :)

Patrick: I have Lua installed and sometimes I write small test programs using lua. But usually I use texlua, because it has all the nice libraries built in (lpeg for example). Actually that is not true, because I have my own patched LuaTeX binary with an XML library built in. I can’t believe that in the year 2012 we don’t have an XML enabled TeX system.

Paulo: I’ve seen some new languages out there with a built-in XML support. :) It’s quite a requirement.

Patrick: It is not hard to write a basic XML parser. But if you really want to do it right (all the different charsets, xinclude and other things) it is getting error prone. So I cannot see a reason why not to include something like libxml2 or expat.

There is now a new XML standard which allows more characters to be used in tag names. I bet that all the self-written parsers will not keep up with these new standards

Paulo: Indeed.

Patrick: But I am arguing with the LuaTeX developers for several years now :)

Paulo: I wrote my first Lua script these days. I’m impressed with Lua’s simplicity. Almost every new software has a piece of code written in Lua.

Patrick: There are some things missing in Lua, for example decent object oriented programming. But if you know the limits, Lua programs are fast to write and readable, even after a few months.

Paulo: Can you name something you really like in LuaTeX? And is there something you dislike? :)

Patrick: One thing I really like is the function tex.linebreak(). It is TeX’s linebreaking algorithm but not with “plain text” as the input, but the data structure (nodes) TeX uses internally.

I dislike that the font system is not built into the core as it is in XeTeX, but this is not so problematic as we thought in the beginning.

egreg: I just want to tell that I’ve enjoyed very much this conversation (and I am safe on the train).

Patrick: Thanks, I also enjoyed the chat.

Paulo: What do you recommend for a newbie eager to learn TeX, LaTeX and friends? :)

Patrick: I don’t know that the state of the art is when it comes to LaTeX introductions. But the second book should be “The LaTeX Companion” by Frank Mittelbach et al.

egreg: I see you’re at 8950: just a four/five days’ work to reach 10000 and get Paulo’s credit card number. ;-) Good job, anyway, I always have a look to your contributions.

Paulo: Oh my credit card number! :D

Patrick: I am mostly interested in LuaTeX contributions. :) I am proud to be the first one to have a bronze “LuaTeX” badge, even before egreg and others have it. :)

But I get my points by answers like these: Does ConTeXt ever require multiple passes to compile? and One-letter word at the end of line.

egreg: Confess that you were inspired by my comment. :)

Patrick: Of course, but it was too verbose ;-)

Paulo: Do you have a “favorite” answer of yours?

Patrick: Yes, I do. Actually two, the “cow in a box one” and the answer about how to prevent a single letter word at the end of a line. These answers show the beauty how Lua and LaTeX can exist in perfect harmony.

A user does not have to know Lua to use the packages, but still get the power of Lua. Both answers are rather short and might provide a starting point to learn how Lua and LaTeX work together.

Paulo: Thanks a million for this great interview! 🙂

Patrick: I have to thank you and the others for the great questions!

---

Stay tuned for the next episode of TeXtalk!