I have to say, any definition of "framework" that claims the general concept of higher-order functions is a subset of frameworks, does not seem particularly useful in the day-to-day.

In this case, `map` is given as an example of a micro-framework. While it does illustrate the author's point, I think all it does is showcase that the separation really isn't as clean as they want it to be, and it undermines rather than reinforces their philosophy.

Swap out “framework” for “inversion of control” in your head and you might find the article more useful. It’s nice seeing these options listed out.

I agree - the author's use of "framework" here betrays the article a bit, and "inversion of control" is a much more accurate portrayal of what they're actually getting at. Of course this is a natural side-effect of what happens when terms are born without proper definitions.

It's almost as if the author has defined "blue" as "any RGB colour where B>0", and then their first example is magenta. Yes, it's a nebulously defined concept (both "blue" and "framework"!), but this means that trying to impose a rigid definition on top is bound to fail.

I’d care more about the misnaming if it was frameworks versus anything else, but it’s not. It’s frameworks vs frameworks. You could call it X and say all the code examples are members of X and I’d find the article just as valuable.

I think this is Grade A software engineering content. Here’s a thing you do sometimes, here’s seven other ways to do it with names, code examples, trade offs and real world examples. It’s great for learning because when you come across this problem in the future you have all your tools laid out for you. I also found it personally helpful because I’m visiting Python framework interface options right now. This saved me a bunch of work.

Still plenty missing, delegate a'la macOS/iOS, singleton/global/envs, explicit parameters/context object, multimethods a'la clojure/miltiple dispatch a'la julia, functors a'la ocaml, plugins/convention-based-autoloading, even monkey patching a'la RoR/active-stuff-style if a form of configruation/inversion of control, probably more.

Do a write up of all the missing patterns in the same style as this article! I’ll read it.

The use of map threw me off, too. The author mentions React as an example of that pattern. Not sure why they didn't use that. Express.js comes to mind, too, as a very grokkable example. (Edit: or perhaps the author considers Express.js an example of an imperative registration API?)

Under "convention over configuration" is this bit:

> pytest also goes further and inspects the arguments to functions to figure out more things.

I think this pattern deserves its own category. I think of it as the Python world's variation on "dependency injection" and I really like it.

In pytest you can use argument names to request that specific test fixtures be made available to your test function: https://docs.pytest.org/en/6.2.x/fixture.html

I use it in Datasette to allow plugins to define their own view functions, which will be passed the specific objects that they declare a need for in order to process an incoming HTTP request: https://docs.datasette.io/en/stable/plugin_hooks.html#regist...

I think people only like that when they write the code themselves and/or are familiar with it.

But if you have to work in a new project / code base and there is a lot of this kind of magic, then it's really really hard to figure out how things work and where you need or even can make changes to change behaviour.

I would go so far to say that this is actually an anti-pattern.

The alternative that I prefer: have a sane default that is generated somewhere and searchable for anyone who doesn't know the convention yet. And in the case of pytest, I think this is just a shortcoming of the language in spite of a better way of doing proper dependency-injection. I remember very well that angular 1.x did something very similar and it was very confusing (they called .toString on the function to find the argument-names). It also violates the general idea of being able to safely refactor a method argument name without breaking things.

The Spring framework (Java) does the same thing for controller methods. If you need an authenticated user, or a model object, or a path variable, just add it to the method signature and the framework will provide it.

I don’t know if there is a term for this concept. I’ve always seen it as some form of Dependency Injection/IoC but at the method level instead of object creation.

IIRC the Actix web framework in Rust does something similar for handler functions.

The problem with doing this in Python is that there is no typing or interfaces to help you. Basing it purely on naming of arguments breaks the assumption that argument names are local to the function/method. I find this extremely confusing to reason about, let alone the possible unwanted side effects if some developer isn’t aware that a name is magical. It’s also very non Pythonic for good reasons.

I feel the same way (and mostly use unittest instead) but maybe this technique makes sense as long as its use is limited to test functions?

Test functions are never called explicitly and would otherwise (like unittest TestCase methods) never have any arguments, so in this context maybe it's clear that any arguments they do have must be magical.

It can actually play really well with Python's optional typing. I should add that to the implementation in Datasette!

I've always just called it "spooooky annotation magic".

The decorator pattern seems to fit, from my POV. The framework takes your code, creates a proxy that implements the interface, and then wraps your method in a method that does the stuff you asked for with the annotations before and after your method gets called.

I think the distinction between frameworks and libraries is much more fuzzy and philosophical than what's presented here. A framework says "Here's how we're going to do things. I'll allow you to extend and build out in certain directions, but you have to use my channels." A library says "Here are some pieces, I don't know or care what you're going to do with them, figure it out." A framework is the Apple philosophy applied to code.

A framework's restrictions can be enforced by IOC, or by integration and compatibility between its subsystems (and incompatibility with alternatives), or just by strong conventions and tutorials/documentation that stick to a beaten path, or any combination of the above. I don't think the particular mechanisms of constraint are as important as the fact that there is constraint.

I’m not sure who put it this succinctly, but I remember it generalized this way: a framework calls your code, but you call the library’s code.

This is inversion-of-control principle, popularized (but not coined) by Martin Fowler in an article of the same name.

GP comment says it's not what defines a framework, it just happens to be a succinct summary of most methods that frameworks use to enforce the their philosophy, which is in GP's view what defines a framework: it has opinions and philosophy about how you structure your code, and it wants you to follow them.

I knew it as the Hollywood Principle: "Don't Call Us, We'll Call You"

A framework calls your code, but in practice the term is loaded with a set of independent characteristics, such as it being the frame of your entire application, not just an aspect of it (libraries can also "call you" in some cases, no one forbids a library from taking in a callback function or an object).

And with that, frameworks often become their own universe, where external components need to be "integrated" with the framework in order to enable using them pragmatically at all. So you either rely on the framework for everything, or you go looking for plugins for the framework, or if you need functionality outside it, it has to be integrated.

Inversion of control is a great principle when used with care. In the hands of amateurs, it's used to just replicate the unit version of a "god object", where your entire application becomes a unit defined by the framework.

You don't need a framework of any kind to do inversion of control though.

I didn't claim otherwise.

This is how the OP put it, and I'm pushing back against this definition.

I may have given a too cursory read on this article but it seems to be confusing the higher level concept of frameworks with the lower level concept of inversion-of-control. While IOC is indeed the usual foundation for frameworks, and thus also all programming approaches to IOC, a framework does not differentiate itself from competitors by those, but through a wide range of capabilities offered to users.

This puts into words why I hate subclass based frameworks and get footgunned by some convention-over-configuration frameworks.

My preference would be function based, interface based and annotation based, in that order.

Interfaces instead of subclassing is clear as an alternative and it has the benefit of allowing multiple inheritance.

It's unclear what "function based" and "annotation based" would mean though.

If you mean passing closures that implement a specific contract, that's in effect "interface based" again. And annotations seem orthogonal in terms of use cases (also full disclosure, I find about 99% of annotation use to be poorly designed and leading to unnecessary static coupling).

The first pattern in the article is function based. Annotations based is called "language integrated registration" in the article.

Not a bad article overall but I wish the author had spent some time talking about error and exception handling.

At some point code you’ve written is going be called by the framework and throw an exception. Recovery is often more difficult or not possible because you don’t have much knowledge of the calling frame. Are any of these approaches better or worse suited, or do they have any special requirements?

I’m thinking of among other things Java Runnables throwing exceptions and silently killing threads.

I don't think there's anything specific to frameworks about exception handling.

You're not supposed to throw exceptions the caller doesn't expect. What does it expect? Well it expects what's documented on the type is takes (either as checked exceptions, or by convention if that's not part of the language).

And any other unexpected errors should be of an appropriate type (Error in Java for ex.) where the framework will finalize its resource handles, and rethrow to some global handler either you or the framework defines. At which point it's in your hand.

And if your exception doesn't fit such a scenario, then probably it shouldn't have been thrown to the framework's stack frames in the first place.

Speaking of frameworks, I don’t know how react gets away with calling itself a library. I think the only reason people call it a library is that it’s subtitle says it’s a library. It requires you to structure a large part of your app in a highly specific way by subclassing or implementing an interface, and react calls your code. It’s not that opinionated, but it’s far more opinionated than express, which is considered a framework (although it’s the lightest framework I’ve ever seen, and basically just procedurally builds a http event listener for you, so it’s very library like)

Well, one way of looking at it is that a framework is just a big (or very big) library. I don't know the history of React, but maybe it began as a somewhat smaller library and then grew more than expected?

> A software framework is code that calls your (application) code.

I see the meaning here, but at risk of bikeshedding, I feel as if it understates the relationship between framework and application.

Rails uses the word 'scaffolding' for a bunch of the stuff it generates for you. In that sense, the framework is pretty much all of the foundation and also the load-bearing support structure for what you're actually building. All of the architecture is in place for you, essentially.

It doesn't just call your application code; it is the application.

And yeah, in that sense... programming languages are themselves frameworks over machine code.

I've come to like a cleaner separation, so I would reach for things in the order of function, interface and then subclassing as the complexity demands it. The downside is I've had a hard time with type hinting and enforcement on callbacks in Python. Ideally I'd like to register a function with a decorator (like in "language integrated registration" or "language integrated declaration") and get highlighting/prompting in PyCharm.

My very first choice, which is typically missed by "modern" [0] languages and approaches, would be to put the stuff in a buffer, and have the client read it out when it is ready.

Callbacks are usually inferior:

• The client must create lots of small functions that need to conform to some strange interface that needs some context parameter type (void * in C or complicated type hackery in other languages).

• Each of those callback functions is harder and more boilerplatey to implement because it's completely broken out of the client's control flow - there is no context around.

All this applies to inheritances / "interface" mechanisms in some languages just as well. I don't know why we still haven't abandonded this crap, it adds nothing but new words and types to achieve the same things.

What do we gain from using simple buffers?

• Better decoupling of client vs library/framework/whatever implementation: temporal decoupling. Client can decide when is the right time to take some action.

• Client can setup the necessary boilerplate in a single function (stack frame) once, and then process all messages, of all types, instead of having redundant boilerplate for each callback.

• No inconvenient context type (void * or whatever) or other conformance to any interface needed.

What can the library do when the buffer is full?

• Simple - it should back out and return to user code. The user needs to process the existing messages in the buffer first. The user should then call into the library and have it reattempt what it did last.

When do simple buffers not work?

• Only when the library needs some immediate reaction to the "event". In other words, if it is inconvenient to implement the library in an event-driven fashion and is better implemented in a completely synchronous fashion and needs feedback from the user. For example, the library might request some memory from the user that must be satisfied immediately because the implementation can't back out of the current function.

I think libraries that requires the user to make callbacks should be a rare exception, and not the norm.

[0] When I read "modern" and I am in a cynical mood, I tend to think "ignorant of the old, simple, and proven ways".

I don't find your first two points convincing. The structure of the data put in the buffer is equivalent to the interface of the function. The buffer does encourage you to use data instead of classes. I want more guidance to my users about the contract that they're expected to fulfill and I don't think buffers help with that problem. I do appreciate the temporal aspect you're talking about. I'd be all about if I was Erlang, but I don't think it's worth it for my purpose in Python.

An example of this would be a pull parser for XML, right? A callback-based API can be built on top of the pull-based one if desired, but not the other way around, as far as I can tell, without using a separate thread. So the pull-based approach (i.e. putting stuff in a buffer) is more flexible.

But the issues you point out around callbacks and passing context parameters apply mostly to C. In languages that support closures it's easy to give callbacks whatever context they need.

You could make a callback that only pushes to a buffer for later processing - to achieve the same behaviour that I suggested. So in a way, callbacks are more general (for an event that was pushed to a buffer, you can't have it be immediately processed by the other party, unless you add major threading and scheduling hackery).

Nevertheless I recommend to never make a callback based API unless there is a clear need for callbacks. If there isn't a need, plain buffers (Maybe I should better say "queues") and decoupled processing are the right thing. Callbacks encourage synchronous processing and where they are used to only push to a buffer they are only overhead.

This was helpful to me, gave it a read and may revisit in the future. I'm always a bit muddled in my comprehension of formal coding theory.

The section on callbacks should probably mention that it's best to allow some sort of context cookie to be passed along to the callback. This allows callbacks to be re-entrant and to target any side effects to some particular context. Compare, e.g. qsort() to qsort_r().

Functions are re-entrant unless they refer to and modify state outside themselves. If you mean recursive calls, that happens rarely in frameworks which deal with a very 'flat' processing pipeline.

No, I don't mean recursive. I mean the callback might need to read (or write) some state that's different than the state used by other, concurrent instances of the callback, or other arbitrary threads, so the framework should provide a means for it to get at that state that doesn't rely on say, global variables, so that multiple instances of the callback may be running concurrently with different state. Typically this is done with a cookie (in C, a "void *" parameter is generally used for this.)

Basically, if you write a framework that has callbacks, and you don't make allowance for passing such a context cookie, you're imposing a constraint on your users that you might not mean to. Maybe "re-entrant" wasn't quite the right word, but in my defense, the qsort_r() man page contains this: "In this way, the comparison function does not need to use global variables to pass through arbitrary arguments, and is therefore reentrant and safe to use in threads."

In addition to allowing one to write re-entrant callbacks, it also allows passing in arbitrary data of whatever kind, not just whatever parameters the framework author happened to think of.

This isn't a problem in any of the languages the article mentions (Python, Ruby, JavaScript, Java) so that might be why the author didn't bring it up. Callbacks in these languages are objects that can carry along their own data.

This is because in C, a function pointer is just a bare pointer to code, and cannot imply extra data such as your cookie. Usually a 'function' can do just that, either by capturing data (a 'closure') or by partial application.