I find very interesting the general idea of re-writing. It's also a complicated problem, when you have more high level programs. What does it mean to rewrite? Usually, for example, as the program becomes more complex, you introduce the distinction between data structures and control structures. And in this case, normally, the re-writing is only happening on the data part.
Yeah, but of course they are intimately linked. Somehow, it's the same thing as you said, it's similar to rewrite the algorithm that processes it. It has a similar effect. It's not so direct, but of course they are so interlinked that there you can't really distinguish.
I mean, you could argue the same speaking of neural networks, because the cell holds a state, and this state evolves. Basically that's a kind of rewriting of the algorithm.
But maybe it could even be applied to the algorithm itself?
Yeah, but for example in a neural network the program continuously changes its data, if you consider "data" the weights, for example. And that actually alters the program, it modifies the neural network behaviour. So it's very difficult to find a clear boundary there.
I mean, there's always a boundary. There is always a little program that evolves another program. And that little program, of course, you define it once, usually. Or you iterate and then you have it, basically. That's also at the core, for example, of why people are not convinced that google page rank is an algorithm, or timeline presentation is an algorithm, because without all the models and data that it uses it's nothing. The algorithm is embedded in all these models that are making up most of the energy that is used. Maybe that's interesting, to think about: how often the symbols are touched. This might help in understanding which one is actually the important part of an algorithm. For example, if you need to traverse all the data a lot of times, it's obvious that the data somehow has a stronger weight in the overall formulation of the algorithm.