tag:blogger.com,1999:blog-4642782805835050446.post7379957968145451327..comments2014-08-05T00:51:46.815-07:00Comments on Martin Sulzmann's Blog: Playing with regular expressions: IntersectionMartin Sulzmannnoreply@blogger.comBlogger16125tag:blogger.com,1999:blog-4642782805835050446.post-24667638093896346602009-05-29T14:57:31.533-07:002009-05-29T14:57:31.533-07:00Thanks for your comments Dan.
Correct. The case "...Thanks for your comments Dan.<br /><br />Correct. The case "r1 == Empty || r2 == Empty = Empty" as suggested by you.<br /><br />The second problem arises because of a bug in the convert function.<br />Conversion of the regular equation<br />1 = (a,1) yields a* which is obviously incorrect.<br /><br />The base case in convert2 needs to be (Empty,Phi) instead of (Empty,Empty). There's also the case dealing with letters missing.<br />I'll update the code on hackage in the next few days.Martin Sulzmannhttp://www.blogger.com/profile/02909988196308862749noreply@blogger.comtag:blogger.com,1999:blog-4642782805835050446.post-15530370142467583762009-05-28T13:26:20.599-07:002009-05-28T13:26:20.599-07:00Thanks for the interesting code and references! I&...Thanks for the interesting code and references! I've been playing with it a bit, and I think I've identified a couple of bugs, one seemingly trivial, the other I'm not sure how to fix.<br /><br />The simple one is in the base cases for intersect. The line "r1 == Empty || r2 == Empty = Empty" implies that the intersection of <> with anything other than {} is <>, but it seems that the intersection of <> and 'a' should be {}, as no input matches both expressions. Another manifestation of this is that the intersection of 'a' and <'a','b'> returns 'a', rather than {}. A simple fix is to test the non-Empty argument with isEmpty, returning Phi if it doesn't match the empty string.<br /><br />The not-so-simple one involves differences in the expression following a kleene star. The intersection of <'a'*,'b'> and <'a'*,'c'> should be {}, but intersect returns (modulo simplification) <'a','a'*>.<br /><br />I can see that taking partial derivatives of the original expressions with respect to 'a' leads to a recursive call with the same arguments, and that replacing that result with a variable somehow throws away the distinction between the expressions, but I don't understand the approach well enough to see a solution.Danhttp://www.blogger.com/profile/11694980728276672755noreply@blogger.comtag:blogger.com,1999:blog-4642782805835050446.post-27758656486150355512009-05-06T23:50:00.000-07:002009-05-06T23:50:00.000-07:00Yes, I am looking for something like that. Where c...Yes, I am looking for something like that. Where can I read more about how to relate two regular expressions? Any existing libraries or examples which can do this?<br /><br />Also I have other questions/thoughts/ideas. But for now the above thing is more important and any direction in this aspect is much appreciated. ThanksGowthamnoreply@blogger.comtag:blogger.com,1999:blog-4642782805835050446.post-20392843384055793852009-05-06T22:40:00.000-07:002009-05-06T22:40:00.000-07:00If R1 doesn't match a string s,
i.e. s not in ...If R1 doesn't match a string s,<br />i.e. s not in L(R1) and<br />R2 <= R1, then R2 doesn't match<br />the string either.<br />That's what you're looking for?Martin Sulzmannhttp://www.blogger.com/profile/02909988196308862749noreply@blogger.comtag:blogger.com,1999:blog-4642782805835050446.post-82160270070088507102009-05-06T13:09:00.000-07:002009-05-06T13:09:00.000-07:00Thanks for reply. So the problem is something like...Thanks for reply. So the problem is something like this in brief. Suppose I have lots and lots of regular expressions to match from a set of documents. I want to do this very fast. <br />Eg: Match R1, R2....Rn over files F1, F2....Fm<br />As far as I can think of, we can do two kinds of improvements<br />1. Have a regex matching engine that is optimized to kind of regexes I pass. Don't worry about this for now.<br />2. Can we optimize the regexes by removing some of them or merging some of them, there by reducing the total time taken to match all these regexes over the set of docs.<br /><br />So with respect to second point (filtering regexes), if I can do some kind of partial ordering OR find some kind of related regexes where I can discard some of them. Eg: If I can say R1 is related to R2 in one way, disregard R2, if R1 does not match. Something on this lines.Gowthamnoreply@blogger.comtag:blogger.com,1999:blog-4642782805835050446.post-23944909065858723162009-05-05T20:15:00.000-07:002009-05-05T20:15:00.000-07:00Thanks for reply. So the problem is something like...Thanks for reply. So the problem is something like this in brief. Suppose I have lots and lots of regular expressions to match from a set of documents. I want to do this very fast. <br />Eg: Match R1, R2....Rn over files F1, F2....Fm<br />As far as I can think of, we can do two kinds of improvements<br />1. Have a regex matching engine that is optimized to kind of regexes I pass. Don't worry about this for now.<br />2. Can we optimize the regexes by removing some of them or merging some of them, there by reducing the total time taken to match all these regexes over the set of docs.<br /><br />So with respect to second point (filtering regexes), if I can do some kind of partial ordering OR find some kind of related regexes where I can discard some of them. Eg: If I can say R1 is related to R2 in one way, disregard R2, if R1 does not match. Something on this lines.Gowthamnoreply@blogger.comtag:blogger.com,1999:blog-4642782805835050446.post-90154066626484768032009-05-05T14:53:00.000-07:002009-05-05T14:53:00.000-07:00"multiple regular expressions to match"
Can you p..."multiple regular expressions to match"<br /><br />Can you pl elaborate, short example?<br /><br />Are you saying you'd like to reuse the partial results of a previous match?Martin Sulzmannhttp://www.blogger.com/profile/02909988196308862749noreply@blogger.comtag:blogger.com,1999:blog-4642782805835050446.post-35150355640392514772009-05-05T14:38:00.000-07:002009-05-05T14:38:00.000-07:00It is a very nice post. Lot helpful. I have some q...It is a very nice post. Lot helpful. I have some questions on how to use the operations/comparisons over regular expressions. I would not say that my question is totally related to this post, but for sure you people will have great ideas.<br /><br />In brief, if I have multiple regular expressions to match, To do it very fast, Can I establish some relationship among them and skip some regular expressions using the relationship. I guess partial ordering does some ordering, but could not find good info about it on net.Gowthamnoreply@blogger.comtag:blogger.com,1999:blog-4642782805835050446.post-27101566492022368782008-11-24T14:29:00.000-08:002008-11-24T14:29:00.000-08:00My PhD student Kenny Lu's forthcoming thesis conta...My PhD student Kenny Lu's forthcoming thesis contains more applications of partial derivatives in the context of regular expression pattern matching. I'll summarize the main results in a blog post, once the thesis is through the entire reviewing process.Martin Sulzmannhttp://www.blogger.com/profile/02909988196308862749noreply@blogger.comtag:blogger.com,1999:blog-4642782805835050446.post-35370509991294753722008-11-23T16:05:00.000-08:002008-11-23T16:05:00.000-08:00Antimirov shows that his partial derivative constr...<I>Antimirov shows that his partial derivative construction leads to fairly 'optimal' DFAs.</I><BR/><BR/>Which makes it all the more wrong to require DFA minimisation for correctness. :-)<BR/><BR/>I suspect that non-minimal DFAs may be more common in lexer generators, where you're matching against multiple regular expressions in parallel. I have no justification for this, though.Pseudonymhttp://www.blogger.com/profile/04272326070593532463noreply@blogger.comtag:blogger.com,1999:blog-4642782805835050446.post-75798441301614010712008-11-22T00:26:00.000-08:002008-11-22T00:26:00.000-08:00I see.Antimirov shows that his partial derivative ...I see.<BR/><BR/>Antimirov shows that his partial derivative construction leads to fairly 'optimal' DFAs.<BR/><BR/>Here are some references:<BR/><BR/>Rewriting Extended Regular Expressions.<BR/> Antimirov, Valentin M. and Mosses<BR/><BR/>Valentin M. Antimirov: Partial Derivatives of Regular Expressions and Finite Automaton ConstructionsMartin Sulzmannhttp://www.blogger.com/profile/02909988196308862749noreply@blogger.comtag:blogger.com,1999:blog-4642782805835050446.post-41578092555759950802008-11-21T21:48:00.000-08:002008-11-21T21:48:00.000-08:00Yes, this code was intended to implement DFAs, rat...Yes, this code was intended to implement DFAs, rather than just compute set intersection.<BR/><BR/>One reason why I like your method better is that the standard algorithm sometimes ends up with "infinite failure". This regular expression, for example:<BR/><BR/>(a|b)* - b*(ab*)*<BR/><BR/>recognises the empty set. The DFA produced looks like this:<BR/><BR/>q0 -> a q0 | b q0<BR/><BR/>where q0 is not a final state. This does indeed accept the empty set, but it only does so after consuming the entire string.<BR/><BR/>Obviously, this behaviour is undesirable in practice. When we wrote the lexer generator for Mercury, we solved the problem by making the subsequent DFA minimisation step do the dirty work. (You put a known "empty set" state with no transitions to or from it into the DFA, then perform state minimisation, then any state which ends up in the same partition as that special state must also be an "empty set" state.)<BR/><BR/>However, that always sat badly with me. DFA minimisation should be an optional optimisation step, rather than required for correctness.Pseudonymhttp://www.blogger.com/profile/04272326070593532463noreply@blogger.comtag:blogger.com,1999:blog-4642782805835050446.post-55047140232315612272008-11-20T00:16:00.000-08:002008-11-20T00:16:00.000-08:00Thanks for the comment Andrew. I had a quick scan ...Thanks for the comment Andrew. I had a quick scan through your code. You follow the 'traditional' approach by converting regular expressions to DFAs?<BR/><BR/>The point of my code is that we can implement standard regular expression operations by rewriting them (without having to via an explicit DFA construction).<BR/><BR/>My post is a response to <BR/>http://www.iis.sinica.edu.tw/~scm/tag/regular-expression/<BR/>I think I've screwed up some of my blogs settings, so Shin couldn't leave a comment.Martin Sulzmannhttp://www.blogger.com/profile/02909988196308862749noreply@blogger.comtag:blogger.com,1999:blog-4642782805835050446.post-88380506024050589412008-11-19T20:40:00.000-08:002008-11-19T20:40:00.000-08:00Here's my version from 2003. I didn't bother impl...Here's <A HREF="http://www.ninebynine.org/Software/HaskellRDF/Dfa/Dfa.lhs" REL="nofollow">my version from 2003</A>. I didn't bother implementing intersection or set difference at the time.<BR/><BR/>Set difference turns out to be slightly more useful in practice than intersection. One example is in C-style comments; they are most naturally defined as "/*", followed by any string not containing "*/", followed by "*/".Pseudonymhttp://www.blogger.com/profile/04272326070593532463noreply@blogger.comtag:blogger.com,1999:blog-4642782805835050446.post-15311267243303557692008-11-08T19:33:00.000-08:002008-11-08T19:33:00.000-08:00Thanks for the interesting post and code, as well ...Thanks for the interesting post and code, as well as references to partial derivatives!<BR/><BR/>Some questions: 1. when you said "to use co-induction which then yields a finite (recursive) proof," do you mean the use of convert?<BR/><BR/>2. Before the definition of convert you said "variables x appears .. at position (r,Var x)," do you mean x appears only at right most positions in the (now a mixture of regular and context-free) grammar? What constraints do we assume on the input of convert, so that we can always convert a regular expression r, possibly with x in it, to r1*r2?<BR/><BR/>Thanks!Shin-Cheng Muhttp://www.iis.sinica.edu.tw/~scmnoreply@blogger.comtag:blogger.com,1999:blog-4642782805835050446.post-29793747660190060922008-11-05T08:08:00.000-08:002008-11-05T08:08:00.000-08:00For Lineage 2 Cheats, Lineage 2 Dupes, Lineage 2 a...<A HREF="http://www.exploitsrus.com/l2.html" REL="nofollow">For Lineage 2 Cheats,</A> <A HREF="http://www.exploitsrus.com/l2/dupes.html" REL="nofollow">Lineage 2 Dupes, Lineage 2 adena cheats,</A> <A HREF="http://www.exploitsrus.com/l2/bots.html" REL="nofollow">Lineage 2 Bots,</A> <A HREF="http://www.exploitsrus.com/l2/guides.html" REL="nofollow">Lineage 2 Guides, and Walkthroughs click here</A>leechsluckershttp://www.blogger.com/profile/08696635963530228892noreply@blogger.com