Code, data and interactive programming

MOV R0, dev_mouth
MOV R1, disgust_string
JMP write
RET
disgust_string:
.asciz "ewww"

when the pointed objects are redefined, it isn’t clear what should happen with the pointing objects. Should they point to > the new thing or the old thing? Either answer can be counter-intuitive to you

Weeeell. I was completely with you until this passage. This passage resembles the following typical sentiment oftentimes uttered by FP weenies: "Side effects are awful! You never know what will change as a result of a side effect. Should the change affect the running code or shoud it not? Either answer can be counter-intuitive to you!". Well: nope. If side-effects are properly contained (or, say, wrapped in transactions or in a dataflow-based consistency-preserving mechanism, or how about just being explicit and clearly labeled?), there is no problem.

Same with redefinition. Redefinition can have clearly defined semantics. Imagine that: it can be that you can know exactly what will happen when you redefine something!

I don't know how things are in the Python world, but in the reportedly autistic and ghettoish but still quite comfortable Lisp micro-universe you have basically two semantic models of redefinition.

One model is based on the concept of symbols. For instance, calling a function by name (as opposed to FUNCALL'ing an object, which is slightly more syntactically cumbersome — the cornerstone of usability is having right defaults, see) always indirects through the symbol function slot (I'm glossing over some insignificant performance-oriented details here, but never mind). This means that when you redefine a function, all the call sites magically switch to using the new definition. Kwadratisch, praSimple, convenient, interaction-friendly.

The other model is used in CLOS. It in turn has two facets:

Class redefinition is explicitly supported, by having all existing instances become of the new class. All of them, effective immediately. The process is customizable through a protocol. The referers, of course, see the changes immediately.

Likewise, you can change the class of a specific instance. This, too, will be immediately reflected everywhere the instance is used, and this process too can be customized to your liking.

The take-home message is, I suppose, that many seemingly scary things can work just fine if they are explicitly designed into the language/environment/whatever.

That said, I do value the ability to restart my Lisp session once in a while. It's not the redefinition semantics that is the problem here, but rather the fact that starting over can often be radically simpler than cleaning the mess up from inside, as it were. Not to mention the need to check whether your stuff will even load up given a clean starting image.

Foist, a bunch of random reactions, and then, to the point.

* Side effects are evil :) Technically, I ain't no FP weenie because everything I actually use has side effects. But when you do compilers or debug stuff, side effects sure are damn evil. It's a problem... I'm still thinking about it.

* Lisp isn't Python or Unix or Matlab; Lisp isn't punk technology (hence the lack of popularity). I'll blog about it some day. Anyway, the fact that CLOS has a "customizable protocol" to help redefine classes during interactive development convinces me again that no, it ain't no punk technology... I think that what happens in Python basically is, everything is an object and when there are no references to it, it's garbage-collected; classes are just one kind of object, and there's no such thing as "redefeinition" – "class A" is basically assignment, just like "A = 5". Makes interactive work a problem, but is very straightforward to implement. Three-chord music. Welcome to the future.

Now, to the point:

First of all, I think that Lisp is less pointer-happy than Python, Perl and their ilk; AFAIK, class objects don't point to methods, for one thing. But, what does #'sym do? Doesn't it give you a handle to use for funcall, and doesn't it force Lisp to keep the old code block around? Or did you actually mean to say just that (funcall effectively calls by pointer, normal calls by name?)

I don't really understand how code works in Lisp; I know (or feel that I know) the C model (where you compile and link everything and don't even dream about "redefining" anything at run time), and the Python model (where code is one kind of object, basically, and you can point to it and the system collects the garbage). Is the Lisp model much different from the Python model? It seems to be, if CLOS can speak about "redefining". Or is it about the same, with CLOS classes simply keeping track of living objects and intercepting redefinition and running code on those objects upon redefinition?

BTW, I think that Python handles function calls just the way Lisp does according to your description: when you call functions by name, it locates the most updated definition at run time, and when you push function references to data structures (the way you'd do with #'sym), it pushes a reference to the definition rather than the name.

So currently the picture to me is:

* Lisp is less pointer-happy than Python because objects don't point to code blocks.
* Lisp is otherwise similar to Python in the sense that you can end up having multiple definitions of the function hanging around; maybe it's not a problem because you don't have that much function-valued variables.

Is this about right?

Lisp isn’t Python or Unix or Matlab; Lisp isn’t punk technology (hence the lack of popularity). I’ll blog about it some day.

Just link to Worse is Better, that's gonna save you the trouble. :)

Anyway, the fact that CLOS has a “customizable protocol” to help redefine classes during interactive development

Well, the protocol is mostly a show-off item (and the result of the fact that the CLOS designers were being rather unusually thorough). Pretty much no practical situation needs it, because things Just Work.

But, what does #’sym do? Doesn’t it give you a handle to use for funcall, and doesn’t it force Lisp to keep the old code block around?

#'sym gives you whatever sits in the function slot of 'sym. You can also use FUNCALL on 'sym itself, though, if you'd like possible redefinitions of the function to be picked up at that particular call site.

Is this about right?

Yes. Indeed, Lisp's runtime model is basically the same as Python's (manifestly-typed objects, lots of pointers, GC, the works); the interactive usability differences are mostly due to peculiarities of the evaluation semantics and the object system.

(I'm not sure how the fact that in Python methods "belong" to classes changes the picture one way or the other, though, but perhaps the understanding will strike me a minute after I submit this comment. Happens all the time, no reason to wait).

If (funcall 'sym) works, what's the point of having #'sym – to get in the way of interactive work? :)

The thing with Python classes that I meant wasn't the single-dispatch thing (one way to interpret "methods belong to classes"), but the fact that objects point to classes and classes point to code blocks, so when you "redefine" classes, the old objects keep pointers to the old code blocks.

Now that I think of it. So CLOS updates all objects upon class redefinition (I wonder how it keeps track of them and what happens if you have threads in the background, but it's just perverse, forget it).

Well, I add a slot to a class vector3d, which previously kept x, y, z, and in this slot I cache the vector magnitude. How will CLOS know to set the new slot to the right value in the old object? What if I had a bug in the magnitude computation code, and now I reload the definition? How will it know to recompute that slot? I mean, how is it going to reestablish class invariants for old instances when class invariants change? I don't think it can be mechanically solved. Is it "really" a non-problem, or just seems to be a non-problem in practice?

The thing is, if you have untyped data, then "it means the same thing all the time", but if you have types and type-based binding, no matter how you do it, it basically means that objects /point/ to code which expects certain things and these things are no longer true and stuff. And it gets in the way of redefining. I think.

If (funcall ’sym) works, what’s the point of having #’sym – to get in the way of interactive work?

I see the smilie, but let me indulge at being boring: there are anonymous functions (made with LAMBDA), there are locally-scoped functions (introduced by FLET etc.) — in short, there is no sense at all to restrict FUNCALL to just symbols.

when you “redefine” classes, the old objects keep pointers to the old code blocks.

Er. Don't they redirect through some sort of vtable that sits in the class object?

Oh yeah. LAMBDAs. Should have thought about those. They ought to be kept in multiple versions, BTW, since you can't redirect via persistent something like a symbol, right?

Python classes – yeah, they redirect via the class dictionary, so instance objects keep indirect pointers to code blocks. Nobody keeps track of the class objects; if something did, it could change the class object without changing all the instance objects.

They ought to be kept in multiple versions, BTW, since you can’t redirect via persistent something like a symbol, right?

Sorry, who ought to be kept in multiple versions by whom?  You mean what happens if you "redefine" an anonymous function?  But you cannot redefine something that is anonymous.  By, um, definition.  You can only make a new object, and perhaps strike it lucky and make sure to change all the references to the old one by references to the new one. :)

If you are gently prodding me to admit that Lisp cannot support every possible redefinition one can think of — well, that's obviously true.  All it does is making sure that when you want something to be redefinable, you can easily have that, and that the syntax and evaluation rules are designed to make the redefinable cases the natural (that is, more concise and readable than the alternative) choice in most cases.

they redirect via the class dictionary, so instance objects keep indirect pointers to code blocks. Nobody keeps track of the class objects; if something did, it could change the class object without changing all the instance objects.

So basically the main missing thing is a way to redefine a class while keeping the identity of the class dictionary?

If you are gently prodding me to admit that Lisp cannot support every possible redefinition one can think of — well, that’s obviously true.

No, I sincerely wanted to make sure I understand what happens :)

So basically the main missing thing is a way to redefine a class while keeping the identity of the class dictionary?

Yep. But then I think you have more complicated problems, that of updating the old instances according to the new class invariants, and I wonder how CLOS can handle that; I sneaked in a comment about it above. The bottom line is that I think type-based binding is basically a way of linking objects to code, and it can make objects "lose meaning" at the logical level no matter how redefinition is handled.

(Found no way to reply to an individual comment.  Oh, and some sort of comment notification thing would be great.  Hey, ever heard of LiveJournal? :))

So CLOS updates all objects upon class redefinition (I wonder how it keeps track of them and what happens if you have threads in the background, but it’s just perverse, forget it).

No "keeping track" needed.  The usual implementation strategy is to check whether the class is current upon each access to an instance, so you get a certain penalty for working with CLOS (which is why frobbing slots in inner loops is not a very good idea) but most of the magic is lazy.

Well, I add a slot to a class vector3d, which previously kept x, y, z, and in this slot I cache the vector magnitude. How will CLOS know to set the new slot to the right value in the old object?

This is where that fancy redefinition protocol comes in.  This margin is sort of too small for the details to fit, but basically you have four main entry points: fresh object initialization, reinitialization, update for redefined class and update for a different class.  You are told what class changed and to what (where applicable), which slots are initialized explicitly, etc.

What if I had a bug in the magnitude computation code, and now I reload the definition? How will it know to recompute that slot? I mean, how is it going to reestablish class invariants for old instances when class invariants change? I don’t think it can be mechanically solved. Is it “really” a non-problem, or just seems to be a non-problem in practice?

I don't know whether this can be objectively measured to be a problem or a non-problem, but of course it's not done by magic.  You need a dataflow mechanism to automate it (such things exist, and even come with convenient CLOS metaclasses).  I think Excel gets this right out of the box.

it basically means that objects /point/ to code which expects certain things and these things are no longer true and stuff. And it gets in the way of redefining. I think.

The general redefinition problem is, of course, intractable.  I think it is not a good reason to give up on interactive programming altogether, because certain kinds of disciplined redefinition are tractable, useful, can be made convenient and happen to cover most of the need.

I guess the redefinition protocol is for the cases where you perform a surgery on a living system that you really don't want to restart – otherwise I'd probably restart the shell at this point... It's cool that it is supported though, for the surgical cases – no good way to do that in most (all?) environments.

So in general the code-update problem is the same as the serialization problem; a brute-force way to do a code update is to serialize your universe and then deserialize it. In fact, that's pretty much the normal way that people upgrade software in the C world.

There's a tradeoff between losing too much state at upgrade time and having to write schema migration scripts for too many classes.

The equivalence works the other way too; if you can locate all of the objects that somebody might want to update to new code, and run a migration script on each one with introspective knowledge of what was in the object before, then you can serialize the objects in the migration script.

Anyway, I think your proposed "pointer-happy systems can't be interactive" rule is bosh. It's true that it can be pretty hard to figure out how an object got into a weird state after you change the code underneath it a few times, but that's a pretty rare frustration, in my limited experience of Squeak and SBCL. Squeak and Erlang are two of the most pointer-happy programming environments, and also two of the most interactive. Elisp is full of objects that refer to each other, but it's radically interactive. Supposedly the old LispM systems were even more interactive without being any less pointer-happy.

I'd like to suggest an alternative explanation. Maybe interactivity is a sine qua non for novice programming environments, and so the best interactive programming environments happen to be novice programming environments; and novices are allergic to pointers. (At least that's what Joel claims, and he helped design one of the best ones: Excel.)

Well, there are different kinds of pointer; I meant, first and foremost, code pointers, which have the biggest redefinition/serialization problems. I don't think Erlang is that pointer-happy in this regard.

I've worked a lot with Matlab, tcsh and IDLE/IPython. The redefinition problems in Python tend to grow as your program grows, unless you bend and twist your style to work around it, and even then it still sucks. Allergic to pointers I hardly am.

In Squeak, almost all of the code pointers are in classes; although it has first-class closures, they almost never live very long. Classes live in a global dictionary, so they're easy to find. When you define a menu item that invokes some arbitrary code, you generally define it by giving it a class and a method name. So changing the definition of a method pretty much just does the right thing.

Similarly, in Lisp, most functions live in some module namespace, and are referred to with an indirection through that namespace (just as in Python). This makes function calling slow but means that you can change what function gets called.

In Erlang, which is a language I don't actually know, I think you're encouraged to pass module names and function names, instead of actual functions, when you define long-running processes, specifically so that if you reload the module, the long-running process will start running the new version of the code after the next time it loops at the top level.

"The newbie realizes that the difference between code and data is trivial. The expert realizes that all code is data. And the true master realizes that all data is code."

http://blogs.msdn.com/sriram/archive/2006/01/15/lisp_is_sin.aspx

It comes down to the fact that Python has no become() method, like smalltalk has, so if you change a class at runtime, new instanciations will find the old class, but already instanciated objects will point to the old code.

So, reloading code works if you access this code through its name (module.attribute) but not otherwise (if a pointer to the old code was kept).

Even so, there are problems :

- What if you update the code for a class, and add a data member ? Are you going to update all the existing objects ? Are you going to create a runtime code versioning scheme ? What if the information to put in this data member is only available at object construction ?

- What if you update 2 different pieces of code that depend on each other and you change the interface between them ? Old bits of this code that still linger and want to call the new code will fail. New code that happens to call an old bit of code will fail. And there is the question of doing the update atomically.

So it looks good in practice, until you consider these evil dependencies.

Reloading code is very easy in Python, I've used it successfully several times, but it only works well in certain circumtances :

- you don't change interfaces, or you reload atomically all the stuff that depends on each other

- whatever contains pointers is short lived and will very soon use the new code

For instance : in a Python web server, you can easily reload the code for a page, and it's essential that you can do so (who would want to restart the server everytime a page is modified ?). Lock all requests for this page (and wait for current requests to finish), reload the code, unlock.

You can't easily reload the web server logic itself, though, because it lives for a long time.

And if you have session objects stored in RAM, you can't reload the code for them either, unless you implement code versioning, or your session objects are very simple mappings (with no code like class instances in them).

I wrote a bittorrent tracker as an asynchronous HTTP server. Some parts of it are reloadable at run-time (error strings, stats pages, the parts that control the policy applicable to users, etc). For simplicity I didn't put this reloadable stuff in classes, they're just a bunch of functions. Since functions are called by name (module.attribute) in Python, and I didn't keep function pointers around, it worked very well.

I wrote this 4 years ago. It could handle more than 1000 HTTP requests per second. Yeah I could have written it in C++, lol, what for ? In Python it took me a week, and once debugged, the thing just runs, never crashes (Python is a managed environment, it will log an exception, not crash). I've left this team since 4 years now, and none of the guys know Python. All they know is how to stop and restart the tracker when there is a server migration, lol. Last time I checked, this python code has been running for 6 months straight without crashing or manual restart. Try that, C++.

In Python, you could pickle (serialize) your entire environment, save it, reload your code, and reload all your environment. Code is not saved : class instances are serialized with the module.classname in them to be able to instantiate them back. If you want to update the code for all current in-memory objects, you could do it like that (and it provides a simple way to save a running session).

Don't wait until PyPy catches up, though, it can serialize a running thread and teleport it to another box...

Oops, I mixed "old" and "new" in some places.

Pickling replaces pointers to code with name-based references, that's what you need I guess.

[...] enter my brain are either data or cigarette smoke, because I don't do drugs. And I hope ... Read More RECOMMENDED BOOKS REVIEWS AND OPINIONS computer abbreviations market research, [...]

Post a comment