ace/api/tokenizer.html

<div id="documentation" class="span8"><div class="classContent"><div class="membersBackground"></div><div class=" members pos0"><div class=" membersContent pos0"><h1 class="memberHeader"><span class="name">Tokenizer<span class="editInC9"> <a href='http://c9.io/open/git/?url=git%3A%2F%2Fgithub.com%2Fajaxorg%2Face.git' title='Edit in Cloud9 IDE'>[edit]</a></span></span></h1><ul data-tabs="tabs" class="nav tabs pos0"><li data-dropdown="dropdown" class="dropdown"><ul class="dropdown-menu"></ul></li><li data-dropdown="dropdown" class="dropdown"><a href="#" data-toggle="dropdown" class="dropdown-toggle"><span>Constructors (1)</span><b class="caret"></b></a><ul class="dropdown-menu"><li id="dropdown_Tokenizer.new" data-id="Tokenizer.new" class="memberLink"><a href="#Tokenizer.new" class="" title="Tokenizer.new (constructor)" data-id="Tokenizer.new">new</a></li></ul></li><li data-dropdown="dropdown" class="dropdown"><a href="#" data-toggle="dropdown" class="dropdown-toggle"><span>Functions (1)</span><b class="caret"></b></a><ul class="dropdown-menu"><li id="dropdown_Tokenizer.getLineTokens" data-id="Tokenizer.getLineTokens" class="memberLink"><a href="#Tokenizer.getLineTokens" class="" title="Tokenizer.getLineTokens (class method)" data-id="Tokenizer.getLineTokens">getLineTokens</a></li></ul></li><li data-dropdown="dropdown" class="dropdown"><ul class="dropdown-menu"></ul></li><li data-dropdown="dropdown" class="dropdown"><ul class="dropdown-menu"></ul></li><li data-dropdown="dropdown" class="dropdown"><ul class="dropdown-menu"></ul></li><li data-dropdown="dropdown" class="dropdown"><ul class="dropdown-menu"></ul></li></ul></div></div><article id="Tokenizer" data-title="Tokenizer (class)" class="article"><div class="section description"><div class="memberContent"><p>This class takes a set of highlighting rules, and creates a tokenizer out of them. For more information, see <a href="https://github.com/ajaxorg/ace/wiki/Creating-or-Extending-an-Edit-Mode#wiki-extendingTheHighlighter">the wiki on extending highlighters</a>.</p>
</div></div></article><h3 class="sectionHeader">Constructors</h3><article id="Tokenizer.new" data-title="Tokenizer.new (constructor)" class="article"><div class="section method"><div class="memberContent"><div class="title"><i id="Tokenizer.new" class="methodToggle methodClicker inactive icon-caret-right"></i><ul class="signatures"><li class="signature"><ul><li class="signature-call"><span class="constructorIdentifier">new </span><span id="Tokenizer.new" class="member-name methodClicker">Tokenizer</span>(<span class="sigArgList"><a href="http://www.nodemanual.org/latest/js_doc/Object.html" class="argument methodClicker" title="Object" data-id="Object">Object</a> rules, <a href="http://www.nodemanual.org/latest/js_doc/String.html" class="argument methodClicker" title="String" data-id="String">String</a> flag</span>)</li></ul><ul class="metaInfo"></ul></li></ul></div><div class="sideToggler"><div id="ellipsis_Tokenizer.new" class="ellipsis_description"><p>Constructs a new tokenizer based on the given rules and flags.</p>
</div><div class="description"><p>Constructs a new tokenizer based on the given rules and flags.</p>
<h4>Arguments</h4><table class="argumentTable argument-list table table-striped table-bordered"><tr class="argumentRow "><td class="argName ">rules</td><td class="argType" "><a href="http://www.nodemanual.org/latest/js_doc/Object.html" class="" title="Object" data-id="Object">Object</a></td><td class="argDescription "><p>Required.  The highlighting rules</p>
</td></tr><tr class="argumentRow "><td class="argName ">flag</td><td class="argType" "><a href="http://www.nodemanual.org/latest/js_doc/String.html" class="" title="String" data-id="String">String</a></td><td class="argDescription "><p>Required.  Any additional regular expression flags to pass (like &quot;i&quot; for case insensitive)</p>
</td></tr></table></div></div></div></div></article><h3 class="sectionHeader">Methods</h3><article id="Tokenizer.getLineTokens" data-title="Tokenizer.getLineTokens (class method)" class="article"><div class="section method"><div class="memberContent"><div class="title"><i id="Tokenizer.getLineTokens" class="methodToggle methodClicker inactive icon-caret-right"></i><ul class="signatures"><li class="signature"><ul><li class="signature-call"><span id="Tokenizer.getLineTokens" class="member-name methodClicker"><span class="sigClassName">Tokenizer.</span><span class="sigMemberName">getLineTokens</span></span>(<span class="sigArgList"></span>)<li class="signature-returns"><ul class="argument-types"><li class="argument-type"><a href="http://www.nodemanual.org/latest/js_doc/Object.html" class="returnType " title="Object" data-id="Object">Object</a></li></ul></li></li></ul><ul class="metaInfo"></ul></li></ul></div><div class="sideToggler"><div id="ellipsis_Tokenizer.getLineTokens" class="ellipsis_description"><p>Returns an object containing two properties: <code>tokens</code>, which contains all the tokens; and <code>state</code>, the current state.</p>
</div><div class="description"><p>Returns an object containing two properties: <code>tokens</code>, which contains all the tokens; and <code>state</code>, the current state.</p>
</div></div></div></div></article></div><script defer src="./resources/javascripts/ux.js"></script><script src="./resources/javascripts/clicker.js"></script><script src="./resources/javascripts/jquery-scrollspy.js"></script><script defer src="./resources/javascripts/disqus-ext.js"></script><script defer src="./resources/javascripts/ga.js"></script><div id="disqus_thread"></div></div>