TBH, the content of your story is good. I mean the plot is something I'd like to try as well, though it might need some work on story progression and coherence. I got how the story is supposed to go, but I guess it's troublesome for the disoriented, can't relate, and those who have no sense of direction, they probably wouldn't get how point A in the story ended up in point B all of a sudden in the next sentence.
A back story is good, or some visualization of what happened could rid the confusion. Try to put yourself in the character's POV and see how you'd want your feelings to get across to "her" (of course you'd want to be clear and concise to get the message across, right?). You could also try to reread the piece from a reader's POV and see if you'd understand the piece without knowing what you already do know as the writer of the piece itself, and if it's pleasant to read. (You could read this out loud too, if it sounds awkward, then there's something wrong. If it's pretty flowy, then you're getting there!)
Hope this helped!