Support cross-stage persistence (CSP)

[LPTK]

Cross-stage persistence is the ability for code fragments to refer to values created at a previous stages, as in val foo = 'ok; code"Some(foo.name)" – notice the bare reference to foo, which is not an inserted piece of code, but really the value 'ok.

This capability is extremely useful when doing runtime multi-stage programming. For instance, imagine that you are making a data manager class DataManager that uses multi-stage programming (i.e., runtime code generation, composition, and compilation) to speed-up some of its operations. It could look like:

abstract class DataManager {
  val data: Array[Int]
  def iterator = data.iterator
  def stagedIterator = code"data.iterator"
}

Using myDataManager.stagedIterator, one can obtain a closed piece of code (of type Code[Iterator[Int],{}]) that can be inserted into a bigger program and optimized, and then runtime-compiled –– the generated bytecode will have a runtime reference to myDataManager.data.

The feature does not make sense for compile-time multi-stage programming or static code generation. I had previously shied away from supporting CSP because of the unclear behavior in this case. Squid now solves it in two ways:

• cross-stage persistence is only enabled if the Base extends CrossStageEnabled, which prevents users from using CSP by mistake;

• if CSP is enabled and one tries to generate static Scala code (as opposed to runtime-compiling it), then Java serialization is used on a best-effort basis to convert cross-stage values to a string representation to be loaded when the generated program is ran;
  for example in val ls = List(1,2,3); val cde = code"ls.map(_+1).sum", cde contains a cross-stage reference to a List(1,2,3). Trying to generate static Scala code from it with base.scalaTree(cde.rep) results in:

_root_.squid.utils.serial.deserialize(
  "rO0ABXNyADJzY2FsYS5jb2xsZWN0aW9uLmltbXV0YWJsZS5MaXN0JFNlcmlhbGl6YXRpb25Qcm94eQAAAAAAAAABAwAAeHBzcgARamF2YS5sYW5nLkludGVnZXIS4qCk94GHOAIAAUkABXZhbHVleHIAEGphdmEubGFuZy5OdW1iZXKGrJUdC5TgiwIAAHhwAAAAAXNxAH4AAgAAAAJzcQB+AAIAAAADc3IALHNjYWxhLmNvbGxlY3Rpb24uaW1tdXRhYmxlLkxpc3RTZXJpYWxpemVFbmQkilxjW/dTC20CAAB4cHg="
).asInstanceOf[scala.collection.immutable.List[scala.Int]]
 .map[scala.Int, scala.collection.immutable.List[scala.Int]](((x$1_0: scala.Int) => x$1_0.$plus(1)))(scala.collection.immutable.List.canBuildFrom[scala.Int])
 .sum[scala.Int](scala.math.Numeric.IntIsIntegral)

[fschueler]

This is great :)

[LPTK]

Thanks, Felix!