jam log - monday:
15:15: Thinking I'll start by setting up nanovg and lua. But first, time for a walk.
17:30, Walk results: Starting with a gui is a waste of time. Now embracing the "data model first" philosophy. I have no idea what I'm about to create, so I must be on the right path.
19:30: Successfully dumped a bunch of ideas into notion. Entering background processing mode now. Probably continuing tomorrow.
jam log - tuesday:
10:00: Data model is fine for now. Starting with the tui. Step 1: Getting familiar with ansi escape codes and console input.
13:37: Boy, that was a ride. Now turning ReadConsoleInputW events into something similar to WM_KEYDOWN/UP and WM_CHAR. Time for a break.
14:30: Well, the tui might not have been that good of an idea. As I'm displaying trees (eg: nested bullet points), I have to do full layout and line wrapping anyway. The only real advantage of the tui would be that the console does the text rendering for me. Except, that is actually a disadvantage, because I don't know how long a string is (I'd like to support unicode & emojis). On top of that, there are all the other tui disadvantages - like lack of vsync, being limited to a grid, etc. I could have come to that conclusion earlier. Spending 3 hours on learning console stuff was a bit of a mistake. Though I have also learned how to set up lua and call into C.
18:40: Trying to be be smarter about my approach. Goal: Render basic tree. Need a window and something to draw. Setting up glfw and nanovg.
19:15: Creating lua bindings for basic path drawing.
19:50: Bindings for text drawing.
20:35: Messing with tree layout and drawing in lua.
21:15: Break time.
21:45: More text/tree layout.
23:20: Successfully rendering a tree. Currently layout is in an "origin is bottom-left" coordinate system. The formulae are pretty ugly. Tomorrow, I'll see how that might be cleaned up and start work on input processing. (I don't like "origin is top-left" because math & 3D graphics are bottom-left)
TLDR: Fell into my usual jam trap of not actually working on the project (rabbit holed on gui). Luckily, I stopped myself with (hopefully) enough time left. Tomorrow, I'll start "fresh" - that is actually start. I'll be using hyperapp, a 420 line js "framework".
--
10:30: Time to get to work. First up today: Multiple node types and clean up.
12:00: Messed with interfaces/inheritance in lua. Time to grab some food!
12:30: So about inheritance... As a language feature, it has two main use cases: Code reuse and interfaces. Combining those two into one feature has some ugly side effects. Say, I am only interested in code reuse and decide to use inheritance, because that's what my language provides. The problem is that I now also inherit the interface, which my type may not actually implement (= "fulfil all constraints of the interface"). This is what the Liskov substitution principle is about (it basically says "don't do that"). To illustrate: If Square inherits Rectangle (of course you'd never do that), Square gets Rectangle's functionality, but must now additionally conform to Rectangle's interface, because Square "is a Rectangle". One of those constraints may be that the width and height can be changed independently, which is of course not true for a square. The way to get around that is to duplicate field definitions and create wrapper procedures - not exactly "clean". Another problem is that code reuse through inheritance is not granular. There is usually no way to say "I want procedure a from X and procedure b from Y". If both "X" and "Y" define versions of "a" and "b", I again need wrappers to resolve the cycle (neither one can be inherited first, as the second one overrides both "a" and "b"). And when it comes to interfaces, inheritance is again not the right tool: "Proper" interfaces only specify properties of a type. They shouldn't provide default implementations, that's what code reuse is for. So basically, code reuse and interfaces should be separate language features. At some point, I'll create a language that explores that. But now, how does that relate to my note app? So far, I've used an inheritance based approach (similar to core/object.lua in rxi's lite). I'm not really happy with it because it's more complicated than I'd like and because it has the above problems (mainly non-granular code reuse). I'll now explore a simpler solution by rewriting the gui. It'll be a retained mode implementation with an immediate interface. Very similar to flutter. I'll also use flutter's layout "algorithm". As I've experimented with that before, I can focus on a solution to the code reuse / interface problem.
14:05: Done with my experiments. I'll just have a helper function to define widget types. It just takes a lua table of handler functions (on-layout, on-paint, on-mouse-*, etc). Handlers that are not listed in the table use the default implementation. The approach is somewhat ad-hoc and very specific to widgets in my gui - and I like that. It's simple and granular. Time for a quick break, then I'll implement that widget type creation helper.
14:40: Back to work on the widgets.
15:30: Defining new widget types was really easy: A widget type is just a "v-table" - so a lua table, used as a meta table on instances (with the table's __index pointing to itself). The "base widget" is also a table, containing the default implementations. For now I'm just using lua's "meta table chain" approach, which is actually a lot more flexible than other inheritance implementations. As meta tables are just lua tables, you can say "My_Widget.on_layout = Your_Widget.on_layout" for granular inheritance. Also created some widget tree helper functions. Time to grab some more food.
15:50: Thinking about the immediate mode interface.
16:05: Will go with a "reactive" interface for now. So still declarative but with callbacks. Should be just fine in lua, as it has closures. Going to implement that now.
16:35: Time for a walk!
17:30: Back to work on the gui.
18:45: Taking some time to reflect on the progress.
19:45: So far, I haven't really worked on the note app itself. That's kinda bad. I just checked out hyperapp (a 420 line js "framework"). It seems pretty decent. I think using the browser is actually a good decision for me, as that will allow me to create a mobile version without too much effort (eg: getting stuff like the ios keyboard trackpad thing working). On top of that, the only "real" gui programming I've done so far was in the browser. So yeah, switching mid-jam seems like a good decision, given the state of the project. Once I have a native gui library that I'm happy with, I can still port the app.
Thanks, for writing about your progress, I feel less lonely ^^.
I don't know if you are familiar with all those languages and frameworks/libs, but it seems you're using a lot of different things without a proper/well know "setup".
Unless your goal is to experiment with languages/libs, I would recommend to use any setup you usually use if you have one, otherwise a lot of time in the jam is spent learning those instead of learning things about your project. For example (I'm not familiar with your project) why are you using C and lua, could you use only C or only lua ? Or JS if that's what you're familiar with. You can change to something else after the jam if needed. Also thinking about other platforms, seems counter productive during a jam to me.
When you've got some time after the jam, I recommend watching this video from Sean Barrett: Advice for writing small programs in C, it's not about C, it's about having an easy way to start coding when you want it, whatever the language you use.
After watching that a few years ago, I started making my own setup and grow my own codebase so that I can start coding in a familiar environment in seconds. It takes time getting the different parts running (e.g. you'll not have code to create a window, or instantiate the graphics api at first) but you can rely on other lib to do that (and replace them if you feel the need for it later). When you start using that regularly it becomes easier and easier to make simple application fast.
Good luck.
Edited by Simon Anciaux
on
Hey, thanks for your feedback!
Note: This response turned into more of a reflection on my current situation. Thanks for getting the ball rolling again on these thoughts. I'm not expecting a response, though wouldn't be mad if someone commented on them.
I indeed do not have a go-to setup. And that's costing me a lot of time (-> today's log :D). On top of that I'm pretty biased toward perfectionism. I am fully aware of both problems and in the process of "recovering", but that just takes time.
Usually, I write c++ and do systems-kind-of programming. For the jam, I wanted to try a dynamic language, as I like the idea of using "loose" language features for experimenting (eg: gc, unstructured data).
My initial plan with lua/C was to primarily use lua for the app and use C for the bindings only. The problem was that I didn't have a gui library, so tried to throw together a custom one in a day, which didn't go too well (would have taken much longer).
When it comes to gui apps, I'm not too experienced. The largest ones I've written were online forms using pure html/js/css. Retained mode really wasn't fun.
In the meantime I've experimented with writing a gui library in c++. I took flutter/react as inspirations, as I think the "reactive" approach is very promising. Aside: Building a gui library without having written many guis is probably not the best idea. But that ties into my next problem: I have so many things I want to do and so little time. Or at least I think I have little time. I'm only 21, so I could spend a year, making stuff at the edge of my comfort zone, instead of way beyond it. In the long run that would probably be faster, too.
Anyway back to the story: So, because I had experience with the browser and knew that there were lots of react-style libraries, I just tried the next best one, which was also small and easy to set up. (I really don't like having a lot of stuff that I don't understand below me: node, webpack, babel, whatever else.) So hyperapp, just being a single js file, looked promsising. Sadly it didn't quite cut it. So I switched to preact. Its documentation is lacking, but it has all the features I want, so I have decided to stick with it.
My goal for the rest of the jam is to actually stick with my current setup and just try to get things working, as best I can.
After the jam, I'll probably force myself to finish a bunch of small entirely imperfect things to level up my skills and build a tool base that I can use for small programs.
(The projects I really want to work on are: A programming language, tackling problems of code reuse and granularity, and a structured ide with live development and embedded, interactive documentation. Well, and there is the whole "new web" thing. I have dabbled with the former two, but never got to anything "showable".)
Of course, these words only touch the surface of all the intricacies of my current situation, but I think they give a good idea of the general theme. I think I'm decent at the extremes of programming/development: The "low level" (using the basic features of a language, solving concrete problems). And the "high level" (design, basically). But my "mid level" is lacking. So that, which ties the other two together. That, which allows you to start, finish, and ship a project.
I have some work to do. Thanks for reading :)
PS: I don't know whether this post and my proposed "solution" is a reflection of my perfectionism, or my introspection.
09:15: Setting up the project.
10:05: It was refreshingly easy to get a basic tree with some hover effects onto the screen. While the web stuff certainly has its problems, just the fact that I've used it before meant that I could get running pretty quickly. I'll go for a nice morning walk to think about what I'm going to do today.
12:00: I should do more morning walks - very peaceful, not many people out. Also grabbed some food. Will now implement the ideas. Starting with a tree editor (selection, add/remove nodes, change structure, etc).
13:45: Alright! Have a basic tree editor prototype. There is multi selection and sub-tree deletion. Taking a quick break.
14:00: One pretty annoying thing about hyperapp is that it doesn't have support for local widget state. There is only one "global" app state. When using widgets, I don't want to have to worry about 1) whether they are stateful and 2) where to put their state. IMO, that's a leaky abstraction. So, I'll now try to understand those 420 lines of uncommented and "cleverly" written js. Then I'll either modify hyperapp to suit my needs or just write my own version.
15:45: Well, I don't think its worth spending the time to fix hyperapp. I've now discovered preact, which aims to be a small version of react. This should really be the last tool switch though - preact has all the stuff that I need. Sucks to be "wasting" this much time. But I guess that's almost unavoidable if you don't yet have a go-to tool. Or can write a small tool in a few hours. Which I'm sure I could do, if I really understood the essentials of immediate/reactive dom wrappers — which I don't. While small, preact is still way too complicated for my needs. I think I'll take a week or two after the jam to learn how I can create my own virtual dom library. Taking a break now, then porting my tree editor prototype to preact.
16:25: Back to work on the tree editor.
18:55: Ported the prototype and messed with keyboard input. Messed indeed, the code needs some major clean up - later.
It's just an opinion: don't think about big picture stuff while programming.
High/low level, immediate/retained, code reuse, react... I try to just program what I need at a moment with my knowledge (for example don't say "I need a gui lib" (unless you use one often), but "I need a textbox and a button, how do I code a simple version of that" and don't try to make it reusable). When you've got something working then you can make an (more) informed decision about high level abstractions, code reuse, what immediate mode versus retained mode means and whether it's pertinent for the exact things you're doing.
But even then, in my experience, it doesn't matter for a long time. Most of the time I need to work on several project to be able to start making a decision about how to approach things. And after the decision, is taken I'll probably change my mind later with future knowledge.
Another note, is when you say "I'm a strong believer in..." or "biased toward perfectionism" (I said similar things in the past), those are things that sort of prevents you from properly assessing situation/problems. I don't know how to explain correctly what I mean, but those ideas sort of guided me toward doing things in a certain way without trying to understand them properly or easily dismissing some aspects because they didn't went my way. Hope that makes a bit of sense.
A summary could be, program like you just learned the basics and don't care about any guidelines.
Replying to leddoo (#25214)
Friday:
Made some decent progress. Nothing interesting to show. Sadly got stuck on rich text editing. The last log entry has more on that.
13:00: Worked out what I want to get done by the end of the jam. Getting to work on that now!
14:20: The rough structure of the ui is in place. There is a list of notes (zettels) on the left, an editor in the middle, and an inspector on the right. When you click on a zettel in the list, the editor switches to that zettel. The inspector doesn't do anything yet. Basic stuff. I'll go enjoy the sun for a while; then I'll add more functionality.
16:05: The most important thing, that I still need to figure out, is how to create a "rich text" editor with inline objects in html. So, I'll get to work on that now.
16:40: Yeah, that's gonna be a bit tricky. The most common approach seems to be "contentEditable". The problem is that I'll have my own data structure representing the rich text. To render it, it has to be turned it into dom nodes. Naively using contentEditable will mutate the dom nodes, and I'd have to effectively parse the dom nodes and turn them back into my representation. That doesn't sound like fun.
My current idea is to use contentEditable purely for cursor movement and selection. Any mutating events (i.e. input) will be caught by event listeners, which block the event's default behavior, translate it into mutations of my representation, and trigger a re-render.
Experimenting with that idea now.
17:50: This seems workable: My rich text representation is turned into dom nodes with annotations (source mappings, if you will). beforeinput let's me prevent all input changes and only process those, which make sense for me. Using getTargetRanges, I can figure out which dom nodes (and their sub-ranges) are affected by an edit. The source info lets me map back to my representation. Taking a break, then exploring that further.
19:15: Back to work on the rich text editor thingy: Implementing the dom markup (source locations) and editing.
20:30: It's a bit complicated, but I'm making progress. Need some quick down time.
20:45: Final session for today. Let's see how far I can get with this text editor!
21:55: Well, not that far :D There are a few nasty problems: Not all inputs are cancellable using beforeinput. And for the most important one ("insertText"), it is undefined whether it can be blocked - yay.
So I thought: Alright, let's not block any input events and just "render to make unwanted input go away." That has two problems: 1) preact is a bit silly and doesn't update the text, as it doesn't know that the dom node's text has changed. The v-node still contains the unchanged string, so the diff against my unchanged string prevents the overwriting of the incorrect dom text. I could probably find a workaround for that. But then there is problem 2: If a span's text is changed programmatically, the cursor jumps to the beginning of that span. I guess I could remember the current cursor position and restore it on each render. But I'm not sure that wouldn't cause flickering.
I'm left unsure what to do about these problems. Maybe I'll have an idea tomorrow.
There is of course the option to use an off-the-shelf rich text editor. But that isn't really an option, because I want to do some pretty "core" things differently.
For example: I'll have the usual way of boldening a range of text using two asterisks. However, unlike in markdown, when typing the second asterisk, the range is stored as "structured information", and the characters are removed from the text. (Yes, this will be a separate undo history entry.) But when the cursor moves back into that boldened range, I want to show the asterisks again. They won't quite be normal characters though: You can drag them around using the mouse to change the range, or delete one of them to delete the range (the boldening and the two virtual asterisks go away). I haven't tried to make an existing solution do that, but I wouldn't expect that to be much easier than creating my own.
It would probably be a good idea to look at some other rich text editor implementations though. Many of them use the dom directly though. As in: They don't try to force the browser's hand like mine (instead they send commands to the browser, which then adds <b> tags for example). Hmm, but somehow these editors have to get the info back out of the dom. Come to think of it, maybe dom parsing isn't that bad of an idea after all.
Edited by leddoo
on
Reason: fix spacing
Ok, there are a few things here. Many of them, I care deeply about - not in the way where you are a blind believer, but in the way, where you're trying to seek truth. I've spent *a lot of time thinking about these things.
Just a quick note: This is not about my jam project. It touches more on the greater picture. Regarding the jam project, I'm now in a relatively decent place. Were it not for that damn rich text editor, I'd even be able to produce something usable by sunday. But hey, maybe I can do it! (For that I do use the "don't care about any guidelines" approach. One factor that I am considering however is that I'll continue to work on it after the jam.)
Sorry this got so long :/ You touched on stuff I'm interested in :D
Let's start with the "simple" one: "I'm a strong believer in..." was poorly worded. I didn't decide: "Hey, that looks interesting, everything should be done that way!" Instead, I came to that conclusion by approaching gui from first principles (~ "like you just learned the basics and don't care about any guidelines"). I reflected on my experience working with retained mode gui libraries, messed with implementing my own ones, and read lots of source code of immediate mode and reactive libraries. Unsurprisingly, I came to the conclusion that no single approach works in all situations. However, the reactive approach works well in most cases. (Why I think that is beyond the scope of this response. Though I'd of course be happy to discuss that.)
Regarding "those are things that sort of prevents you from properly assessing situation/problems ...": Yes, I know exactly what you mean (or at least I'm pretty sure that I'm interpreting that correctly). Our models of the world and our assumptions severely limit the options we consider. However, the more aware of that face one is, the more they can overcome it.
Anyway, on to the messy bit:
The approach you describe isn't new to me. In fact, one could maybe even call it "the handmade approach". (Though that would be massively oversimplifying)
The thing is: I cannot find evidence that it works - or at least not for the things that I want to do.
What do I mean by that?
For example: "I need a textbox and a button, how do I code a simple version of that" just doesn't work in the long run. A text box specifically is one of the hardest things to get right, if you're doing it from scratch (there is all the fun that unicode is, cursor movements, text shaping, etc). Now, do you need a good text box for prototyping? Of course not. Or do you even need a unicode text box for the final product? Perhaps not.
But let's assume that you do want a unicode text box in the final product (eg: because you're making a note taking app; and your users expect to be able to use emojis - man those things are complicated). You don't really want to reinvent that over and over. And that really touches on the core of the problem: Making "rich" software (unicode support; high quality ui in general) is incredibly difficult today. Unless you use the browser! (Which is one of the reasons so many companies choose to do that.) There just aren't any decent libraries that abstract over all the complications (and perhaps that's not possible, but I don't think so). Even if you try to use the libraries that browsers use (eg: harfbuzz), you still need a ton of domain knowledge to make a "proper" text box (I've tried).
Of course, this whole thing isn't about text boxes. However they are a perfect example: There are some problems that are practically solved (like text boxes), but there is just no way to benefit from that fact. You can't just extract chrome's text box and put it into your small application (related to the OOP gorilla & jungle problem). And so there is this artificial dichotomy: You can't write small and "rich" apps.
This is getting a bit long, and it's 11 pm, so I'll cut it off here. I hope I could point at the problem precisely enough. Again, it's not about text boxes or GUI, those are merely good examples.
In the end, this probably comes down to philosophy and ones view of computing. Personally, I think computers are "artificially difficult". And I want to enable more people to benefit from computers, which is why I think about these problems so much.
It’s been interesting to catch up on this! I hope you’ve been enjoying the jam overall.
I definitely get your feelings about robust UI for small programs. I wish it were easier to drop in a textbox and have all the controls and Unicode rendering we’ve come to expect. It’s definitely a problem that needs to be addressed.
But, also - it is just a jam! Even crappy text input, or limiting to just ASCII with bitmap fonts, should still allow you to explore your actual high-level ideas. Maybe it’s not the tech you’d actually use if you had more time. But the virtue of a jam is that the code is temporary - you should expect to throw it away afterward.
We’re hitting the last two days of the jam now, so I encourage you to just program for screenshots and video. Make it look how you want it to look even if that means absolutely horrific code. Get those ideas out there! I want to see your ideas! 😄
Hey, thanks for the encouragement!
The jam has been an absolute blast for me. Literally. It has pushed and pulled me in ways I didn't expect.
Were it not for the "official" project announcement and the fact that this community is so supportive, I would have probably quit again. I'm glad that I haven't.
For me, the jam is already a great success. It has made me question where I really stand and what I should focus on. I'm still not quite sure what the answers are, but it feels like an important time to find out.
As for my project: Don't get your hopes up too high :P I wouldn't say that my ideas are game changing. For now, I just want a note taking system that combines the best parts of notion and obisidian with a little bit of my own spice.
I'm hoping to get one of the more interesting ideas implemented by tomorrow evening, but I can't do more than put in the time. I'll definitely continue to work on the project after the jam though.
Thanks again to you and all the other people who put together this awesome jam!
Replying to bvisness (#25233)
penultimate log entry!
mostly did styling and prepared the text editor for tomorrow.
10:25: Trying to apply Ben's suggestion today ("so I encourage you to just program for screenshots and video"). The whole concept of building something and then throwing it away is still super alien to me. I don't really see what it is about, yet. Maybe I will after the jam.
Definitely explains my struggles with jams though. I've always approached them in a way where the goal was to have something polished and usable by the end. Yes, I know that it doesn't really make sense. If you could produce something high quality in such a short amount of time, why wouldn't you always do things that way? In other news, sleeping on the rich text editor hasn't really revealed any new solutions, but I have discovered more problems... This will have to wait until after the jam.
So far, this morning, I've thought about which ideas I'd really want to demo and what I'd need to implement them. Before I get started though, I'll grab some food and go for a walk to think this through, so I don't overcomplicate things.
11:05: Really though, it's not just jam projects. Most of my personal projects are giving me a pretty rough time. I think there are at least two reasons for this: Firstly, many of them are "ambitious" and therefore hard by nature. But secondly, they're my projects and I really, sincerely want to do a good job on them.
I'm probably setting myself up for failure, or at least a hard time, by having such high standards. "Well, then just don't have such high standards, duh." Yeah, except it's not that simple. How are you going to create something of high quality without high standards? Do you ignore them to get started? But, don't you then build a bad foundation that you'll need to replace the moment it is done? Sure, you gain experience when building something bad. But so do you when trying to create something good - and failing because you don't have enough experience - and throwing it away. The end result is the same. Maybe you can iterate faster when not worrying about quality. Maybe it is better for your mental health. But don't you miss things when intentionally making something "not good"?
I do feel like the "don't worry" approach is the way to go. You need to fall before you walk. And every time I've done things that way, I made much more progress in a shorter amount of time. Not uncommonly I realized that my initial approach was misguided - to say the least (which is probably what Simon alluded to).
Why then do I always forget?
12:10: Up until now, the plan had been to make a minimal, but usable note taking app. It wouldn't really have been the one I wanted to make, though.
This whole thing also reminded me of Jon's games: In his interviews, he often mentions that Braid and The Witness had been playable after a few days or weeks. There's a youtube video of an early version of Braid, which literally looked like the a mini version of final game. Pretty much everything was there. Some things were only "seeds", but you could see that it was the same game. I wonder how he does that?
Although I know that I'll have to rewrite it after the jam, I'll now try to create a mini version of my note taking system.
14:50: Alright, did some UI work and planning. Time for some much needed shopping and cooking.
19:40: Back to work on text editing. Trying out an idea for a semi rich text editor.
21:40: Ok, that worked, well enough anyway. Tomorrow I'll add zettel references and "the canvas".
thought it'd be nice to add a screenshot:
final entry!
this is what a somewhat productive day looks like :D
and there's this: https://youtu.be/VxjLTERtDaw
11:05: Final day! Getting to work on the canvas now.
12:15: Infrastructure is in place. Basic panning and zooming are working. Food time!
13:15: Ok, let's add some life to that canvas!
14:15: Stuff can be moved. Nodes are on the canvas. Short break.
14:20: Some infrastructure rework, then references.
15:20: Many small improvements. Zettels are on the canvas, though rendering is messed up.
15:40: Did some planning. Going for a walk in the wind 🍂
17:00: Throwing in more stuff.
18:15: Taking a break.
18:25: Adding a semi-useful feature: Creating new notes.
19:20: The hacks are slowly getting out of hand :D Grabbing food.
20:00: I'm really looking forward to calling this thing done. But before I do that, one more feature: "Turn into Zettel."
20:35: Calling it done!
Very nice!, although it's quite different from my approach to what I've been doing in my note taking system. My main focus has been developing a serialization format (like markdown) that allows embedding structured data in the "note" files.
I'm also working on a textual serialization format for storing just the structured data, with hopefully more compact syntax and a simpler parser than XML, while being more flexible than JSON. I was surprised to see Metadesk because my format looks oddly similar!. I think there's a lot of potential for using a graph interface like the one you're working on, to navigate and create these structured data files.
What I've seen when developing structured data through notes, is that I tend to "tag" notes with words. Say I'm writing notes reviewing books I read, I'll add a "book" tag. Then later, I usually think "I'd like to store the author of everything I've tagged with book" which makes me transform the tag into the type of an object with an "author" property. I think this is a very natural way for structured data to emerge from semi-structured data like tags.
The tag nesting you mention then starts to appear when I want to group multiple similar tags. Say I also had an "article" tag, then I would like to put the "author" property in a more abstract type like "media" of which "book" and "article" are subtypes.
I'm aware that this starts to tread the slippery slope of OOP. Although I don't think OOP is good for structuring software, I've started to realize that the model isn't that bad for structuring data. We just need to develop good tools to easily transform this data, instead of just using text files (like OOP does now a days). Your prototype looks like a good step in that direction!.