Separating words problem

<h2 id="example">Example</h2>
The two strings 0010 and 1000 may be distinguished from each other by a three-state automaton in which the transitions from the start state go to two different states, both of which are terminal in the sense that subsequent transitions from these two states always return to the same state. The state of this automaton records the first symbol of the input string. If one of the two terminal states is accepting and the other is rejecting, then the automaton will accept only one of the strings 0010 and 1000. However, these two strings cannot be distinguished by any automaton with fewer than three states.<a class="footnote-ref" id="fnref:1" href="#fn:1">1</a>

<h2 id="simplifying-assumptions">Simplifying assumptions</h2>
For proving bounds on this problem, it may be assumed without loss of generality that the inputs are strings over a two-letter alphabet. For, if two strings over a larger alphabet differ then there exists a <a href="/facts/String_operations/08G9Cevy">string homomorphism</a> that maps them to binary strings of the same length that also differ. Any automaton that distinguishes the binary strings can be translated into an automaton that distinguishes the original strings, without any increase in the number of states.<a class="footnote-ref" id="fnref:2" href="#fn:2">2</a>
It may also be assumed that the two strings have equal length. For strings of unequal length, there always exists a <a href="/facts/Prime_number/L20FLkgn">prime number</a> p whose value is logarithmic in the smaller of the two input lengths, such that the two lengths are different <a href="/facts/Modular_arithmetic/0wtRa5dU">modulo</a> p. An automaton that counts the length of its input modulo p can be used to distinguish the two strings from each other in this case. Therefore, strings of unequal lengths can always be distinguished from each other by automata with few states.<a class="footnote-ref" id="fnref:3" href="#fn:3">3</a>

<h2 id="history-and-bounds">History and bounds</h2>
The problem of bounding the size of an automaton that distinguishes two given strings was first formulated by Goralčík & Koubek (1986), who showed that the automaton size is always sublinear.<a class="footnote-ref" id="fnref:4" href="#fn:4">4</a> Later, Robson (1989) proved the upper bound O(n2/5(log n)3/5) on the automaton size that may be required.<a class="footnote-ref" id="fnref:5" href="#fn:5">5</a> This was improved by Chase (2020) to O(n1/3(log n)7).<a class="footnote-ref" id="fnref:6" href="#fn:6">6</a><a class="footnote-ref" id="fnref:7" href="#fn:7">7</a> 
There exist pairs of inputs that are both binary strings of length n for which any automaton that distinguishes the inputs must have size Ω(log n). Closing the gap between this lower bound and Chase's upper bound remains an open problem. <a href="/facts/Jeffrey_Shallit/4bmJgyIH">Jeffrey Shallit</a> has offered a prize of 100 British pounds for any improvement to Robson's upper bound.<a class="footnote-ref" id="fnref:8" href="#fn:8">8</a>

<h2 id="special-cases">Special cases</h2>
Several special cases of the separating words problem are known to be solvable using few states:

<ul><li>If two binary words have differing numbers of zeros or ones, then they can be distinguished from each other by counting their <a href="/facts/Hamming_weight/dJIW7qg4">Hamming weights</a> modulo a prime of logarithmic size, using a logarithmic number of states. More generally, if a pattern of length k appears a different number of times in the two words, they can be distinguished from each other using O(k log n) states.<a class="footnote-ref" id="fnref:9" href="#fn:9">9</a></li>
<li>If two binary words differ from each other within their first or last k positions, they can be distinguished from each other using k + O(1) states. This implies that <a href="/facts/Almost_all/KvHxUCCc">almost all</a> pairs of binary words can be distinguished from each other with a logarithmic number of states, because only a polynomially small fraction of pairs have no difference in their initial O(log n) positions.<a class="footnote-ref" id="fnref:10" href="#fn:10">10</a></li>
<li>If two binary words have <a href="/facts/Hamming_distance/MgdtbYPo">Hamming distance</a> d, then there exists a prime p with p = O(d log n) and a position i at which the two strings differ, such that i is not equal modulo p to the position of any other difference. By computing the <a href="/facts/Parity_(mathematics)/7zq2UM4V">parity</a> of the input symbols at positions congruent to i modulo p, it is possible to distinguish the words using an automaton with O(d log n) states.<a class="footnote-ref" id="fnref:11" href="#fn:11">11</a></li></ul>

<h2 id="references">References</h2>

<ol>
<li id="fn:1">Demaine, Erik D.; Eisenstat, Sarah; Shallit, Jeffrey; Wilson, David A. (2011), "Remarks on separating words", Descriptional Complexity of Formal Systems: 13th International Workshop, DCFS 2011, Gießen/Limburg, Germany, July 25-27, 2011, Proceedings, Lecture Notes in Computer Science, vol. 6808, Heidelberg: Springer-Verlag, pp. 147–157, arXiv:1103.4513, doi:10.1007/978-3-642-22600-7_12, ISBN 978-3-642-22599-4, MR 2910373, S2CID 6959459. <a href="978-3-642-22599-4" target="_blank">978-3-642-22599-4</a> <a href="#fnref:1" class="footnote-back-ref">↩</a></li>
<li id="fn:2">Demaine, Erik D.; Eisenstat, Sarah; Shallit, Jeffrey; Wilson, David A. (2011), "Remarks on separating words", Descriptional Complexity of Formal Systems: 13th International Workshop, DCFS 2011, Gießen/Limburg, Germany, July 25-27, 2011, Proceedings, Lecture Notes in Computer Science, vol. 6808, Heidelberg: Springer-Verlag, pp. 147–157, arXiv:1103.4513, doi:10.1007/978-3-642-22600-7_12, ISBN 978-3-642-22599-4, MR 2910373, S2CID 6959459. <a href="978-3-642-22599-4" target="_blank">978-3-642-22599-4</a> <a href="#fnref:2" class="footnote-back-ref">↩</a></li>
<li id="fn:3">Demaine, Erik D.; Eisenstat, Sarah; Shallit, Jeffrey; Wilson, David A. (2011), "Remarks on separating words", Descriptional Complexity of Formal Systems: 13th International Workshop, DCFS 2011, Gießen/Limburg, Germany, July 25-27, 2011, Proceedings, Lecture Notes in Computer Science, vol. 6808, Heidelberg: Springer-Verlag, pp. 147–157, arXiv:1103.4513, doi:10.1007/978-3-642-22600-7_12, ISBN 978-3-642-22599-4, MR 2910373, S2CID 6959459. <a href="978-3-642-22599-4" target="_blank">978-3-642-22599-4</a> <a href="#fnref:3" class="footnote-back-ref">↩</a></li>
<li id="fn:4">Goralčík, P.; Koubek, V. (1986), "On discerning words by automata", Automata, Languages and Programming: 13th International Colloquium, Rennes, France, July 15–19, 1986, Proceedings, Lecture Notes in Computer Science, vol. 226, Berlin: Springer-Verlag, pp. 116–122, doi:10.1007/3-540-16761-7_61, ISBN 978-3-540-16761-7, MR 0864674. <a href="978-3-540-16761-7" target="_blank">978-3-540-16761-7</a> <a href="#fnref:4" class="footnote-back-ref">↩</a></li>
<li id="fn:5">Robson, J. M. (1989), "Separating strings with small automata", Information Processing Letters, 30 (4): 209–214, doi:10.1016/0020-0190(89)90215-9, MR 0986823. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:5" class="footnote-back-ref">↩</a></li>
<li id="fn:6">Chase, Z. (2020), "A New Upper Bound for Separating Words", arXiv:2007.12097 [math.CO]. <a href="/wiki/ArXiv_(identifier)" target="_blank">/wiki/ArXiv_(identifier)</a> <a href="#fnref:6" class="footnote-back-ref">↩</a></li>
<li id="fn:7">Chase, Zachary (2021-06-15). "Separating words and trace reconstruction". Proceedings of the 53rd Annual ACM SIGACT Symposium on Theory of Computing. STOC 2021. New York, NY, USA: Association for Computing Machinery. pp. 21–31. doi:10.1145/3406325.3451118. ISBN 978-1-4503-8053-9. <a href="978-1-4503-8053-9" target="_blank">978-1-4503-8053-9</a> <a href="#fnref:7" class="footnote-back-ref">↩</a></li>
<li id="fn:8">Shallit, Jeffrey (2014), "Open Problems in Automata Theory: An Idiosyncratic View", British Colloquium for Theoretical Computer Science (BCTCS 2014), Loughborough University (PDF). <a href="/wiki/Jeffrey_Shallit" target="_blank">/wiki/Jeffrey_Shallit</a> <a href="#fnref:8" class="footnote-back-ref">↩</a></li>
<li id="fn:9">Demaine, Erik D.; Eisenstat, Sarah; Shallit, Jeffrey; Wilson, David A. (2011), "Remarks on separating words", Descriptional Complexity of Formal Systems: 13th International Workshop, DCFS 2011, Gießen/Limburg, Germany, July 25-27, 2011, Proceedings, Lecture Notes in Computer Science, vol. 6808, Heidelberg: Springer-Verlag, pp. 147–157, arXiv:1103.4513, doi:10.1007/978-3-642-22600-7_12, ISBN 978-3-642-22599-4, MR 2910373, S2CID 6959459. <a href="978-3-642-22599-4" target="_blank">978-3-642-22599-4</a> <a href="#fnref:9" class="footnote-back-ref">↩</a></li>
<li id="fn:10">Demaine, Erik D.; Eisenstat, Sarah; Shallit, Jeffrey; Wilson, David A. (2011), "Remarks on separating words", Descriptional Complexity of Formal Systems: 13th International Workshop, DCFS 2011, Gießen/Limburg, Germany, July 25-27, 2011, Proceedings, Lecture Notes in Computer Science, vol. 6808, Heidelberg: Springer-Verlag, pp. 147–157, arXiv:1103.4513, doi:10.1007/978-3-642-22600-7_12, ISBN 978-3-642-22599-4, MR 2910373, S2CID 6959459. <a href="978-3-642-22599-4" target="_blank">978-3-642-22599-4</a> <a href="#fnref:10" class="footnote-back-ref">↩</a></li>
<li id="fn:11">Demaine, Erik D.; Eisenstat, Sarah; Shallit, Jeffrey; Wilson, David A. (2011), "Remarks on separating words", Descriptional Complexity of Formal Systems: 13th International Workshop, DCFS 2011, Gießen/Limburg, Germany, July 25-27, 2011, Proceedings, Lecture Notes in Computer Science, vol. 6808, Heidelberg: Springer-Verlag, pp. 147–157, arXiv:1103.4513, doi:10.1007/978-3-642-22600-7_12, ISBN 978-3-642-22599-4, MR 2910373, S2CID 6959459. <a href="978-3-642-22599-4" target="_blank">978-3-642-22599-4</a> <a href="#fnref:11" class="footnote-back-ref">↩</a></li>
</ol>

Separating words problem open-in-new

Separating words problem