I just got my handmade RNN (Recursive Neural Network) that uses BPTT (BackProp Through Time) to train and the Adam optimizer, fully working, and it can remember! I am very excited to replace my old RNN impl in my Brainfuck optimizer to this one, to hopefully get amazing results.

In this case (in the screenshot), it detects if a sequence of inputs, over time, are alternating or not (1 if it is repeating, 0 if it is not)