I've been working with a library for my recent project and have ran into a few issues. I believe the issues have arisen because I'm programming everything else from a pretty low level and this library obviously abstracts some of the low level stuff from me for which I now realize I need to get to and integrate with other aspects of my project. This requires me to know more about how that library works and more about the problem space for which that library works in. This got me thinking about the usefulness of libraries and how/when you should really consider to use them for a project, and really what the main purpose of them is.
I saw a clip of Casey discussing exactly this and in that clip he talks about how it's important to understand the problem space the library is working in before using the library. If you're not sufficiently versed in the problem space then you won't be sure if a particular library is good or bad for your project because you won't know if that particular library will integrate well with your project or if you are even using it correctly.
While I tend to agree with his sediments I'm a bit confused now on how I should approach libraries. I know for myself, in order to really sufficiently understand a problem space for a programming project I find it's best to try and program a solution myself from scratch. This way, I gather intimate knowledge that would otherwise elude me and this knowledge is what proves useful for properly understanding what it is I need. Of course, if this is the case then why even use a library if I'm just going to go through the work of implementing and learning everything I need to know myself.
Before hearing Casey I thought the main purpose of a library was to avoid having to go through the work of properly understanding everything in order to use some functionality. Or is it that you just need to be able to find the right balance between understanding yourself and letting a library handle some of that understanding for your? Or is it like Casey says and, at least right now, most libraries aren't developed well enough yet that we can generally use what we need and forget the rest given poorly designed abstractions and bugs? Under what circumstances to you choose a library for a project and what was your thinking behind that choice?
I saw a clip of Casey discussing exactly this and in that clip he talks about how it's important to understand the problem space the library is working in before using the library. If you're not sufficiently versed in the problem space then you won't be sure if a particular library is good or bad for your project because you won't know if that particular library will integrate well with your project or if you are even using it correctly.
While I tend to agree with his sediments I'm a bit confused now on how I should approach libraries. I know for myself, in order to really sufficiently understand a problem space for a programming project I find it's best to try and program a solution myself from scratch. This way, I gather intimate knowledge that would otherwise elude me and this knowledge is what proves useful for properly understanding what it is I need. Of course, if this is the case then why even use a library if I'm just going to go through the work of implementing and learning everything I need to know myself.
Before hearing Casey I thought the main purpose of a library was to avoid having to go through the work of properly understanding everything in order to use some functionality. Or is it that you just need to be able to find the right balance between understanding yourself and letting a library handle some of that understanding for your? Or is it like Casey says and, at least right now, most libraries aren't developed well enough yet that we can generally use what we need and forget the rest given poorly designed abstractions and bugs? Under what circumstances to you choose a library for a project and what was your thinking behind that choice?
Edited by Jason
on
Reason: Initial post
All large, sane projects uses some kind of abstraction, even if that abstraction is just a C-compiler.
For me, ANY abstraction, needs to save me from spending time on things I do not want to do, while still doing that thing about the same way I would do it my self if I did it from scratch.
For me, SDL och SDL_ttf are good libraries, since they make it easy for me to open windows, render, handle events, and render fonts. I do not want to have to sped my time at MSDN.
For the same reason I would never use a string library, since I want to power to control my strings my self.
For me, ANY abstraction, needs to save me from spending time on things I do not want to do, while still doing that thing about the same way I would do it my self if I did it from scratch.
For me, SDL och SDL_ttf are good libraries, since they make it easy for me to open windows, render, handle events, and render fonts. I do not want to have to sped my time at MSDN.
For the same reason I would never use a string library, since I want to power to control my strings my self.
If a library solves your problem, go ahead and use it.
Libraries tend to be more robust than your own code and handle edge cases better. Largely because they are used in more contexts than your code and the bugs other people have run into got fixed. This is especially important in areas where specialized knowledge is required to avoid non-obvious pitfalls like crypto and high performance computing.
And sometimes you just don't have time to implement everything you need simply because it's a tedious bit of work (like all the stuff a gui needs)
Libraries tend to be more robust than your own code and handle edge cases better. Largely because they are used in more contexts than your code and the bugs other people have run into got fixed. This is especially important in areas where specialized knowledge is required to avoid non-obvious pitfalls like crypto and high performance computing.
And sometimes you just don't have time to implement everything you need simply because it's a tedious bit of work (like all the stuff a gui needs)
I'd say that libraries are largely for doing things that are outside your domain of expertise. How you choose or use a library changes based on how well you understand the problem space.
When you're a beginner, you have to use libraries if you want to do anything non-trivial. Unfortunately, you can't evaluate them at all, so you have to rely on popularity, and once you choose a library you typically have to structure your code to fit the library.
If you're an expert in the specific problem space, you can evaluate libraries pretty accurately, but you also have the option to do it yourself (presumably an expert would already be aware of any weird edge-cases and would be able to write a good solution on the first try). At that point you're not using a library in order to do something you'd not be able to do otherwise (as we said, an expert could write their own), you're mostly looking to save time. Now you need to compare the time it would take you to implement it yourself against the time it would take you to evaluate the available libraries. You also need to take into account that any library you might find will probably not be a perfect fit for your specific use-case, whereas rolling your own probably would be.
You'll usually find yourself somewhere in the middle, where you can partially evaluate a library and you can't quite predict how long it would take to implement your own, or how difficult it would be to get it right. In those cases I like to instead evaluate how much I'm going to depend on that library. Or to put it another way: How difficult would it be to replace it with a different library or to write my own at a later point. The more you have to depend on a library, the more you should try to understand the problem space so that you can make a good choice (and to use the library in the best possible way), but that's not always a reasonable way to spend your time, so sometimes you just have to roll the dice and see what happens.
It's very common to encounter the kind of issues where you want to do something that the library doesn't quite provide, or the library doesn't mesh too well with your current code structure (and the docs are probably terrible), so you find yourself "fighting the library" and finding creative workaround for those issues. The "solution" is to become an expert in that problem space and not use those libraries.
In short, you need to choose which rabbit-holes you want to get into, and over time make sure you build a good understanding of your platform and whatever else is important for the kind of code you're writing.
When you're a beginner, you have to use libraries if you want to do anything non-trivial. Unfortunately, you can't evaluate them at all, so you have to rely on popularity, and once you choose a library you typically have to structure your code to fit the library.
If you're an expert in the specific problem space, you can evaluate libraries pretty accurately, but you also have the option to do it yourself (presumably an expert would already be aware of any weird edge-cases and would be able to write a good solution on the first try). At that point you're not using a library in order to do something you'd not be able to do otherwise (as we said, an expert could write their own), you're mostly looking to save time. Now you need to compare the time it would take you to implement it yourself against the time it would take you to evaluate the available libraries. You also need to take into account that any library you might find will probably not be a perfect fit for your specific use-case, whereas rolling your own probably would be.
You'll usually find yourself somewhere in the middle, where you can partially evaluate a library and you can't quite predict how long it would take to implement your own, or how difficult it would be to get it right. In those cases I like to instead evaluate how much I'm going to depend on that library. Or to put it another way: How difficult would it be to replace it with a different library or to write my own at a later point. The more you have to depend on a library, the more you should try to understand the problem space so that you can make a good choice (and to use the library in the best possible way), but that's not always a reasonable way to spend your time, so sometimes you just have to roll the dice and see what happens.
It's very common to encounter the kind of issues where you want to do something that the library doesn't quite provide, or the library doesn't mesh too well with your current code structure (and the docs are probably terrible), so you find yourself "fighting the library" and finding creative workaround for those issues. The "solution" is to become an expert in that problem space and not use those libraries.
In short, you need to choose which rabbit-holes you want to get into, and over time make sure you build a good understanding of your platform and whatever else is important for the kind of code you're writing.
AsafG
The more you have to depend on a library, the more you should try to understand the problem space so that you can make a good choice (and to use the library in the best possible way)
I think this really stood out for me. I read an article that said it is often best to only use libraries for more peripheral things or things that won't be central and specific to your project. I think your quote is sort of the next step in that idea which is you can still use a library for something more central to your project, but if you do it would be best if you truly understood that problem space first so you can still, hopefully, use that library appropriately. I think the main problem I ran into with my current project is that I picked a library to help me out with a central aspect of the program without really understanding that aspect fully, having never programmed it before. In this situation it might've been better to do more of a compression oriented approach from scratch, so that I could learn the problem space more sufficiently along the way, which would also give me the knowledge to integrate things more smoothly as each issue arose.
Edited by Jason
on