{"id":89,"date":"2015-05-28T00:47:47","date_gmt":"2015-05-28T00:47:47","guid":{"rendered":"http:\/\/jeffreymbinder.net\/?p=89"},"modified":"2024-04-13T17:00:08","modified_gmt":"2024-04-13T17:00:08","slug":"the-depoeticizer","status":"publish","type":"post","link":"https:\/\/jeffreymbinder.net\/?p=89","title":{"rendered":"The Depoeticizer"},"content":{"rendered":"

UPDATE 2020<\/b>: I have since made a much more advanced program along these lines<\/a> using neural networks.<\/p>\n

The Depoeticizer changes a text to bring it more in line with the expectations created by a statistical language model. Specifically, it uses an ngram model<\/a>\u2014the same kind that underlies both autocorrect and Markov-chain nonsense generators like the Coleridge Bot<\/a>. But instead of using this model to generate new text, the Depoeticizer uses it to “correct” an existing text, altering words by one or two characters so as to bring the text closer to the norm. It is, in effect, a spell-checker gone berzerk, which can be especially useful if you want to make a poem look more like the language of AP news feeds and financial newspapers.<\/p>\n

(This app is currently offline pending a code update.)<\/p>\n

This program includes four of the Continuous Speech Recognition-III models that are available here<\/a>. It works by assuming that the text was generated based on this model, but that it has a certain chance of containing “typos.” Based on this, it (approximately) reconstructs what the text is most likely to look like without the “typos.” The higher the typo probability, the more likely the program is to change something. A probability of 0.5 means that the program will have no preference between keeping things the way they are and changing them; a probability above that means that it will prefer changing things to keeping them the same.<\/p>\n

The source code for this program is on GitHub<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"

UPDATE 2020: I have since made a much more advanced program along these lines using neural networks. The Depoeticizer changes a text to bring it more in line with the expectations created by a statistical language model. Specifically, it uses an ngram model\u2014the same kind that underlies both autocorrect and Markov-chain nonsense generators like the … Read more<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2,12],"tags":[],"class_list":["post-89","post","type-post","status-publish","format-standard","hentry","category-fun-things","category-posts"],"_links":{"self":[{"href":"https:\/\/jeffreymbinder.net\/index.php?rest_route=\/wp\/v2\/posts\/89","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/jeffreymbinder.net\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/jeffreymbinder.net\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/jeffreymbinder.net\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/jeffreymbinder.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=89"}],"version-history":[{"count":2,"href":"https:\/\/jeffreymbinder.net\/index.php?rest_route=\/wp\/v2\/posts\/89\/revisions"}],"predecessor-version":[{"id":455,"href":"https:\/\/jeffreymbinder.net\/index.php?rest_route=\/wp\/v2\/posts\/89\/revisions\/455"}],"wp:attachment":[{"href":"https:\/\/jeffreymbinder.net\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=89"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/jeffreymbinder.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=89"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/jeffreymbinder.net\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=89"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}