The latest version of tokenify is 0.1.2.0-0.
tokenify
Version 0.1.0.0 revision 2 uploaded by akst.
Package meta
- Synopsis
- A regex based LR tokenizer
- Description
A LR tokenizier used to split text into tokens
- Author
- Angus Thomsen
- Bug reports
- n/a
- Category
- parser
- Copyright
- n/a
- Homepage
- https://github.com/AKST/tokenify
- Maintainer
- aksthomsen@gmail.com
- Package URL
- n/a
- Stability
- n/a