The general idea of re-writing I find it very interesting. It's an interesting question, because it's also a complicated problem when you have more high level programs. What re-writing could mean? Usually, as the program becomes more complex, you introduce the distinction between data structures and control structures. And then the re-writing is only happening usually on the data part.
Yeah, but of course they are intimately linked. Somehow, it's the same thing as you said, it's similar to rewrite the algorithm that processes it. It has a similar effect. It's not so direct, but of course they are so interlinked that there you can't really distinguish.
I mean, the neural networks you could argue the same. Because the cell holds a state, and this state evolves. Then basically that's a rewriting of the algorithm.
But maybe it could even be applied to the algorithm itself or?
Yeah, but for example in a neural network the program continuously changes its data, if it's data is the weights, for example. And that changes actually the program, it modifies the neural network itself. So it's very difficult to find a clear boundary there.
I mean, you always have a boundary there. There is this little program that evolves the other program. And that little program of course, you define it once usually. Or iterate and then you have it, basically. That's also at the core, for example, of why people are not convinced that google page rank is an algorithm or timeline presentation is an algorithm, because without all the models and data that it uses it's nothing. So the algorithm is embedded in all these models that are making out the most of the energy that is used. Maybe that's interesting, to think about how often the symbols are touched. To decide which is actually the important part of an algorithm. So, if you need to traverse all the data a lot of times, it's obvious that the data somehow has a stronger weight in the overall formulation of the algorithm. And data that is separate and data that is just an intermediate representation, like a register in an assembly program. You have symbols that communicate just between the different commands. But then you have other symbols that are stored externally.