mrpastewart/platform
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Migrated symbol_table

Browse source
This commit is contained in:
thomashii@dukefirehawk.com 2021-04-28 07:58:38 +08:00
parent 99c0bd9ada
commit e546ee2bd7
11 changed files with 765 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1
CHANGELOG.md
View file

@ -18,6 +18,7 @@
* Migrated angel_client to 4.0.0 (6/13 tests passed) * Migrated angel_client to 4.0.0 (6/13 tests passed)
* Migrated angel_websocket to 4.0.0 (2/3 tests passed) * Migrated angel_websocket to 4.0.0 (2/3 tests passed)
* Updated test to 4.0.0 (1/1 test passed) * Updated test to 4.0.0 (1/1 test passed)
* Added symbol_table and migrated to 2.0.0 (16/16 tests passed)
* Updated jael to 3.0.0 (in progress) * Updated jael to 3.0.0 (in progress)
* Updated jael_preprocessor to 3.0.0 (in progress) * Updated jael_preprocessor to 3.0.0 (in progress)
* Updated angel_jael to 3.0.0 (in progress) * Updated angel_jael to 3.0.0 (in progress)

17
packages/symbol_table/CHANGELOG.md Normal file
View file

@ -0,0 +1,17 @@
## 1.0.4
* Added `context` to `SymbolTable`.
## 1.0.3
* Converted `Visibility` into a `Comparable` class.
* Renamed `add` -> `create`, `put` -> `assign`, and `allVariablesOfVisibility` -> `allVariablesWithVisibility`.
* Added tests for `Visibility` comparing, and `depth`.
* Added `uniqueName()` to `SymbolTable`.
* Fixed a typo in `remove` that would have prevented it from working correctly.
## 1.0.2
* Added `depth` to `SymbolTable`.
* Added `symbolTable` to `Variable`.
* Deprecated the redundant `Constant` class.
* Deprecated `Variable.markAsPrivate()`.
* Added the `Visibility` enumerator.
* Added the field `visibility` to `Variable`.

21
packages/symbol_table/LICENSE Normal file
View file

@ -0,0 +1,21 @@
MIT License
Copyright (c) 2017 Tobe O
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

158
packages/symbol_table/README.md Normal file
View file

@ -0,0 +1,158 @@
# symbol_table
[![Pub](https://img.shields.io/pub/v/symbol_table.svg)](https://pub.dartlang.org/packages/symbol_table)
[![build status](https://travis-ci.org/thosakwe/symbol_table.svg)](https://travis-ci.org/thosakwe/symbol_table)
A generic symbol table implementation in Dart, with support for scopes and constants.
The symbol tables produced by this package are hierarchical (in this case, tree-shaped),
and utilize basic memoization to speed up repeated lookups.
# Variables
To represent a symbol, use `Variable`. I opted for the name
`Variable` to avoid conflict with the Dart primitive `Symbol`.
```dart
var foo = new Variable<String>('foo');
var bar = new Variable<String>('bar', value: 'baz');
// Call `lock` to mark a symbol as immutable.
var shelley = new Variable<String>('foo', value: 'bar')..lock();
foo.value = 'bar';
shelley.value = 'Mary'; // Throws a StateError - constants cannot be overwritten.
foo.lock();
foo.value = 'baz'; // Also throws a StateError - Once a variable is locked, it cannot be overwritten.
```
## Visibility
Variables are *public* by default, but can also be marked as *private* or *protected*. This can be helpful if you are trying
to determine which symbols should be exported from a library or class.
```dart
myVariable.visibility = Visibility.protected;
myVariable.visibility = Visibility.private;
```
# Symbol Tables
It's easy to create a basic symbol table:
```dart
var mySymbolTable = new SymbolTable<int>();
var doubles = new SymbolTable<double>(values: {
'hydrogen': 1.0,
'avogadro': 6.022e23
});
// Create a new variable within the scope.
doubles.create('one');
doubles.create('one', value: 1.0);
doubles.create('one', value: 1.0, constant: true);
// Set a variable within an ancestor, OR create a new variable if none exists.
doubles.assign('two', value: 2.0);
// Completely remove a variable.
doubles.remove('two');
// Find a symbol, either in this symbol table or an ancestor.
var symbol = doubles.resolve('one');
// Find OR create a symbol.
var symbol = doubles.resolveOrCreate('one');
var symbol = doubles.resolveOrCreate('one', value: 1.0);
var symbol = doubles.resolveOrCreate('one', value: 1.0, constant: true);
```
# Exporting Symbols
Due to the tree structure of symbol tables, it is extremely easy to
extract a linear list of distinct variables, with variables lower in the hierarchy superseding their parents
(effectively accomplishing variable shadowing).
```dart
var allSymbols = mySymbolTable.allVariables;
```
We can also extract symbols which are *not* private. This helps us export symbols from libraries
or classes.
```dart
var exportedSymbols = mySymbolTable.allPublicVariables;
```
It's easy to extract symbols of a given visibility:
```dart
var exportedSymbols = mySymbolTable.allVariablesWithVisibility(Visibility.protected);
```
# Child Scopes
There are three ways to create a new symbol table:
## Regular Children
This is what most interpreters need; it simply creates a symbol table with the current symbol table
as its parent. The new scope can define its own symbols, which will only shadow the ancestors within the
correct scope.
```dart
var child = mySymbolTable.createChild();
var child = mySymbolTable.createChild(values: {...});
```
### Depth
Every symbol table has an associated `depth` attached to it, with the `depth` at the root
being `0`. When `createChild` is called, the resulting child has an incremented `depth`.
## Clones
This creates a scope at the same level as the current one, with all the same variables.
```dart
var clone = mySymbolTable.clone();
```
## Forked Scopes
If you are implementing a language with closure functions, you might consider looking into this.
A forked scope is a scope identical to the current one, but instead of merely copying references
to variables, the values of variables are copied into new ones.
The new scope is essentially a "frozen" version of the current one.
It is also effectively orphaned - though it is aware of its `parent`, the parent scope is unaware
that the forked scope is a child. Thus, calls to `resolve` may return old variables, if a parent
has called `remove` on a symbol.
```dart
var forked = mySymbolTable.fork();
var forked = mySymbolTable.fork(values: {...});
```
# Creating Names
In languages with block scope, oftentimes, identifiers will collide within a global scope.
To avoid this, symbol tables expose a `uniqueName()` method that simply attaches a numerical suffix to
an input name. The name is guaranteed to never be repeated within a specific scope.
```dart
var name0 = mySymbolTable.uniqueName('foo'); // foo0
var name1 = mySymbolTable.uniqueName('foo'); // foo1
var name2 = mySymbolTable.uniqueName('foo'); // foo2
```
# `this` Context
Many languages handle a sort of `this` context that values within a scope may
optionally be resolved against. Symbol tables can easily set their context
as follows:
```dart
void foo() {
mySymbolTable.context = thisContext;
}
```
Resolution of the `context` getter functions just like a symbol; if none is
set locally, then it will refer to the parent.
```dart
void bar() {
mySymbolTable.context = thisContext;
expect(mySymbolTable.createChild().createChild().context, thisContext);
}
```

316
packages/symbol_table/lib/src/symbol_table.dart Normal file
View file

@ -0,0 +1,316 @@
library symbol_table;
import 'package:collection/collection.dart' show IterableExtension;
part 'variable.dart';
part 'visibility.dart';
/// A hierarchical mechanism to hold a set of variables, which supports scoping and constant variables.
class SymbolTable<T> {
final List<SymbolTable<T>> _children = [];
final Map<String, Variable<T>?> _lookupCache = {};
final Map<String, int> _names = {};
final List<Variable<T>> _variables = [];
int _depth = 0;
T? _context;
SymbolTable<T>? _parent, _root;
/// Initializes an empty symbol table.
///
/// You can optionally provide a [Map] of starter [values].
SymbolTable({Map<String, T> values: const {}}) {
if (values.isNotEmpty == true) {
values.forEach((k, v) {
_variables.add(Variable<T>._(k, this, value: v));
});
}
}
/// Returns the nearest context this symbol table belongs to. Returns `null` if none was set within the entire tree.
///
/// This can be used to bind values to a `this` scope within a compiler.
T? get context {
SymbolTable<T>? search = this;
while (search != null) {
if (search._context != null) return search._context;
search = search._parent;
}
return null;
}
/// Sets a local context for values within this scope to be resolved against.
void set context(T? value) {
_context = value;
}
/// The depth of this symbol table within the tree. At the root, this is `0`.
int get depth => _depth;
/// Returns `true` if this scope has no parent.
bool get isRoot => _parent == null;
/// Gets the parent of this symbol table.
SymbolTable<T>? get parent => _parent;
/// Resolves the symbol table at the very root of the hierarchy.
///
/// This value is memoized to speed up future lookups.
SymbolTable<T>? get root {
if (_root != null) return _root;
SymbolTable<T> out = this;
while (out._parent != null) out = out._parent!;
return _root = out;
}
/// Retrieves every variable within this scope and its ancestors.
///
/// Variable names will not be repeated; this produces the effect of
/// shadowed variables.
///
/// This list is unmodifiable.
List<Variable<T>> get allVariables {
List<String> distinct = [];
List<Variable<T>> out = [];
void crawl(SymbolTable<T> table) {
for (var v in table._variables) {
if (!distinct.contains(v.name)) {
distinct.add(v.name);
out.add(v);
}
}
if (table._parent != null) crawl(table._parent!);
}
crawl(this);
return new List<Variable<T>>.unmodifiable(out);
}
/// Helper for calling [allVariablesWithVisibility] to fetch all public variables.
List<Variable<T>> get allPublicVariables {
return allVariablesWithVisibility(Visibility.public);
}
/// Use [allVariablesWithVisibility] instead.
@deprecated
List<Variable<T>> allVariablesOfVisibility(Visibility visibility) {
return allVariablesWithVisibility(visibility);
}
/// Retrieves every variable of the given [visibility] within this scope and its ancestors.
///
/// Variable names will not be repeated; this produces the effect of
/// shadowed variables.
///
/// Use this to "export" symbols out of a library or class.
///
/// This list is unmodifiable.
List<Variable<T>> allVariablesWithVisibility(Visibility visibility) {
List<String> distinct = [];
List<Variable<T>> out = [];
void crawl(SymbolTable<T> table) {
for (var v in table._variables) {
if (!distinct.contains(v.name) && v.visibility == visibility) {
distinct.add(v.name);
out.add(v);
}
}
if (table._parent != null) crawl(table._parent!);
}
crawl(this);
return new List<Variable<T>>.unmodifiable(out);
}
Variable<T>? operator [](String name) => resolve(name);
void operator []=(String name, T value) {
assign(name, value);
}
void _wipeLookupCache(String key) {
_lookupCache.remove(key);
_children.forEach((c) => c._wipeLookupCache(key));
}
/// Use [create] instead.
@deprecated
Variable<T> add(String name, {T? value, bool? constant}) {
return create(name, value: value, constant: constant);
}
/// Create a new variable *within this scope*.
///
/// You may optionally provide a [value], or mark the variable as [constant].
Variable<T> create(String name, {T? value, bool? constant}) {
// Check if it exists first.
if (_variables.any((v) => v.name == name))
throw new StateError(
'A symbol named "$name" already exists within the current context.');
_wipeLookupCache(name);
Variable<T> v = new Variable._(name, this, value: value);
if (constant == true) v.lock();
_variables.add(v);
return v;
}
/// Use [assign] instead.
@deprecated
Variable<T> put(String name, T value) {
return assign(name, value);
}
/// Assigns a [value] to the variable with the given [name], or creates a new variable.
///
/// You cannot use this method to assign constants.
///
/// Returns the variable whose value was just assigned.
Variable<T> assign(String name, T value) {
return resolveOrCreate(name)..value = value;
}
/// Removes the variable with the given [name] from this scope, or an ancestor.
///
/// Returns the deleted variable, or `null`.
///
/// *Note: This may cause [resolve] calls in [fork]ed scopes to return `null`.*
/// *Note: There is a difference between symbol tables created via [fork], [createdChild], and [clone].*
Variable<T>? remove(String name) {
SymbolTable<T>? search = this;
while (search != null) {
var variable = search._variables.firstWhereOrNull((v) => v.name == name);
if (variable != null) {
search._wipeLookupCache(name);
search._variables.remove(variable);
return variable;
}
search = search._parent;
}
return null;
}
/// Finds the variable with the given name, either within this scope or an ancestor.
///
/// Returns `null` if none has been found.
Variable<T>? resolve(String name) {
var v = _lookupCache.putIfAbsent(name, () {
var variable = _variables.firstWhereOrNull((v) => v.name == name);
if (variable != null)
return variable;
else if (_parent != null)
return _parent?.resolve(name);
else
return null;
});
if (v == null) {
_lookupCache.remove(name);
return null;
} else {
return v;
}
}
/// Finds the variable with the given name, either within this scope or an ancestor.
/// Creates a new variable if none was found.
///
/// If a new variable is created, you may optionally give it a [value].
/// You can also mark the new variable as a [constant].
Variable<T> resolveOrCreate(String name, {T? value, bool? constant}) {
var resolved = resolve(name);
if (resolved != null) return resolved;
return create(name, value: value, constant: constant);
}
/// Creates a child scope within this one.
///
/// You may optionally provide starter [values].
SymbolTable<T> createChild({Map<String, T> values: const {}}) {
var child = SymbolTable(values: values);
child
.._depth = _depth + 1
.._parent = this
.._root = _root;
_children.add(child);
return child;
}
/// Creates a scope identical to this one, but with no children.
///
/// The [parent] scope will see the new scope as a child.
SymbolTable<T> clone() {
var table = SymbolTable<T>();
table._variables.addAll(_variables);
table
.._depth = _depth
.._parent = _parent
.._root = _root;
_parent?._children.add(table);
return table;
}
/// Creates a *forked* scope, derived from this one.
/// You may provide starter [values].
///
/// As opposed to [createChild], all variables in the resulting forked
/// scope will be *copies* of those in this class. This makes forked
/// scopes useful for implementations of concepts like closure functions,
/// where the current values of variables are trapped.
///
/// The forked scope is essentially orphaned and stands alone; although its
/// [parent] getter will point to the parent of the original scope, the parent
/// will not be aware of the new scope's existence.
SymbolTable<T> fork({Map<String, T> values: const {}}) {
var table = SymbolTable<T>();
table
.._depth = _depth
.._parent = _parent
.._root = _root;
table._variables.addAll(_variables.map((Variable v) {
Variable<T> variable = new Variable._(v.name, this, value: v.value);
variable.visibility = v.visibility;
if (v.isImmutable) variable.lock();
return variable;
}));
return table;
}
/// Returns a variation on the input [name] that is guaranteed to never be repeated within this scope.
///
/// The variation will the input [name], but with a numerical suffix appended.
/// Ex. `foo1`, `bar24`
String uniqueName(String name) {
int count = 0;
SymbolTable? search = this;
while (search != null) {
if (search._names.containsKey(name)) count += search._names[name]!;
search = search._parent;
}
_names.putIfAbsent(name, () => 0);
var n = _names[name];
if (n != null) {
n++;
_names[name] = n;
}
return '$name$count';
}
}

51
packages/symbol_table/lib/src/variable.dart Normal file
View file

@ -0,0 +1,51 @@
part of symbol_table;
/// Holds an immutable symbol, the value of which is set once and only once.
@deprecated
class Constant<T> extends Variable<T> {
Constant(String name, T value) : super._(name, null, value: value) {
lock();
}
}
/// Holds a symbol, the value of which may change or be marked immutable.
class Variable<T> {
final String name;
final SymbolTable<T>? symbolTable;
Visibility visibility = Visibility.public;
bool _locked = false;
T? _value;
Variable._(this.name, this.symbolTable, {T? value}) {
_value = value;
}
/// If `true`, then the value of this variable cannot be overwritten.
bool get isImmutable => _locked;
/// This flag has no meaning within the context of this library, but if you
/// are implementing some sort of interpreter, you may consider acting based on
/// whether a variable is private.
@deprecated
bool get isPrivate => visibility == Visibility.private;
T? get value => _value;
void set value(T? value) {
if (_locked)
throw new StateError(
'The value of constant "$name" cannot be overwritten.');
_value = value;
}
/// Locks this symbol, and prevents its [value] from being overwritten.
void lock() {
_locked = true;
}
/// Marks this symbol as private.
@deprecated
void markAsPrivate() {
visibility = Visibility.private;
}
}

31
packages/symbol_table/lib/src/visibility.dart Normal file
View file

@ -0,0 +1,31 @@
part of symbol_table;
/// Represents the visibility of a symbol.
///
/// Symbols may be [public], [protected], or [private].
/// The significance of a symbol's visibility is semantic and specific to the interpreter/compiler;
/// this package attaches no specific meaning to it.
///
/// [Visibility] instances can be compared using the `<`, `<=`, `>`, and `>=` operators.
/// The evaluation of the aforementioned operators is logical;
/// for example, a [private] symbol is *less visible* than a [public] symbol,
/// so [private] < [public].
///
/// In a nutshell: [private] < [protected] < [public].
class Visibility implements Comparable<Visibility> {
static const Visibility private = const Visibility._(0);
static const Visibility protected = const Visibility._(1);
static const Visibility public = const Visibility._(2);
final int _n;
const Visibility._(this._n);
bool operator >(Visibility other) => _n > other._n;
bool operator >=(Visibility other) => _n >= other._n;
bool operator <(Visibility other) => _n < other._n;
bool operator <=(Visibility other) => _n <= other._n;
@override
int compareTo(Visibility other) {
return _n.compareTo(other._n);
}
}

1
packages/symbol_table/lib/symbol_table.dart Normal file
View file

@ -0,0 +1 @@
export 'src/symbol_table.dart';

11
packages/symbol_table/pubspec.yaml Normal file
View file

@ -0,0 +1,11 @@
dependencies:
collection: ^1.15.0-nullsafety.4
name: symbol_table
version: 2.0.0
description: A generic symbol table implementation in Dart, with support for scopes and constants.
author: Tobe O <thosakwe@gmail.com>
homepage: https://github.com/thosakwe/symbol_table
environment:
sdk: '>=2.12.0 <3.0.0'
dev_dependencies:
test: ^1.15.7

14
packages/symbol_table/symbol_table.iml Normal file
View file

@ -0,0 +1,14 @@
<?xml version="1.0" encoding="UTF-8"?>
<module type="WEB_MODULE" version="4">
<component name="NewModuleRootManager" inherit-compiler-output="true">
<exclude-output />
<content url="file://$MODULE_DIR$">
<excludeFolder url="file://$MODULE_DIR$/.pub" />
<excludeFolder url="file://$MODULE_DIR$/build" />
</content>
<orderEntry type="inheritedJdk" />
<orderEntry type="sourceFolder" forTests="false" />
<orderEntry type="library" name="Dart SDK" level="project" />
<orderEntry type="library" name="Dart Packages" level="project" />
</component>
</module>

144
packages/symbol_table/test/all_test.dart Normal file
View file

@ -0,0 +1,144 @@
import 'package:symbol_table/symbol_table.dart';
import 'package:test/test.dart';
main() {
late SymbolTable<int> scope;
setUp(() {
scope = SymbolTable<int>(values: {'one': 1});
});
test('starter values', () {
expect(scope['one']?.value, 1);
});
test('add', () {
var two = scope.create('two', value: 2);
expect(two.value, 2);
expect(two.isImmutable, isFalse);
});
test('put', () {
var one = scope.resolve('one');
var child = scope.createChild();
var three = child.assign('one', 3);
expect(three.value, 3);
expect(three, one);
});
test('private', () {
var three = scope.create('three', value: 3)
..visibility = Visibility.private;
expect(scope.allVariables, contains(three));
expect(
scope.allVariablesWithVisibility(Visibility.private), contains(three));
expect(scope.allPublicVariables, isNot(contains(three)));
});
test('protected', () {
var three = scope.create('three', value: 3)
..visibility = Visibility.protected;
expect(scope.allVariables, contains(three));
expect(scope.allVariablesWithVisibility(Visibility.protected),
contains(three));
expect(scope.allPublicVariables, isNot(contains(three)));
});
test('constants', () {
var two = scope.create('two', value: 2, constant: true);
expect(two.value, 2);
expect(two.isImmutable, isTrue);
expect(() => scope['two'] = 3, throwsStateError);
});
test('lock', () {
expect(scope['one']?.isImmutable, isFalse);
scope['one']!.lock();
expect(scope['one']?.isImmutable, isTrue);
expect(() => scope['one'] = 2, throwsStateError);
});
test('child', () {
expect(scope.createChild().createChild().resolve('one')!.value, 1);
});
test('clone', () {
var child = scope.createChild();
var clone = child.clone();
expect(clone.resolve('one'), child.resolve('one'));
expect(clone.parent, child.parent);
});
test('fork', () {
var fork = scope.fork();
scope.assign('three', 3);
expect(scope.resolve('three'), isNotNull);
expect(fork.resolve('three'), isNull);
});
test('remove', () {
var one = scope.remove('one')!;
expect(one.value, 1);
expect(scope.resolve('one'), isNull);
});
test('root', () {
expect(scope.isRoot, isTrue);
expect(scope.root, scope);
var child = scope
.createChild()
.createChild()
.createChild()
.createChild()
.createChild()
.createChild()
.createChild();
expect(child.isRoot, false);
expect(child.root, scope);
});
test('visibility comparisons', () {
expect([Visibility.private, Visibility.protected],
everyElement(lessThan(Visibility.public)));
expect(Visibility.private, lessThan(Visibility.protected));
expect(Visibility.protected, greaterThan(Visibility.private));
expect(Visibility.public, greaterThan(Visibility.private));
expect(Visibility.public, greaterThan(Visibility.protected));
});
test('depth', () {
expect(scope.depth, 0);
expect(scope.clone().depth, 0);
expect(scope.fork().depth, 0);
expect(scope.createChild().depth, 1);
expect(scope.createChild().createChild().depth, 2);
expect(scope.createChild().createChild().createChild().depth, 3);
});
test('unique name', () {
expect(scope.uniqueName('foo'), 'foo0');
expect(scope.uniqueName('foo'), 'foo1');
expect(scope.createChild().uniqueName('foo'), 'foo2');
expect(scope.createChild().uniqueName('foo'), 'foo2');
var child = scope.createChild();
expect(child.uniqueName('foo'), 'foo2');
expect(child.uniqueName('foo'), 'foo3');
expect(child.createChild().uniqueName('foo'), 'foo4');
});
test('context', () {
scope.context = 24;
expect(scope.context, 24);
expect(scope.createChild().context, 24);
expect(scope.createChild().createChild().context, 24);
var child = scope.createChild().createChild()..context = 35;
expect(child.context, 35);
expect(child.createChild().context, 35);
expect(child.createChild().createChild().context, 35);
expect(scope.context, 24);
});
}