Hi Kevin,
Thanks for using 3DNA and for your kind words regarding the 3DNA software! It has always been a pleasure to do something meaningful to myself and helpful to the community. Over the years, it is users like you that help move the project forward. So I always welcome any (technical) questions regarding various aspects of 3DNA.
Now back to your questions. Two issues need to be clarified.
Firstly, the base-pair reference frames as output from 3DNA are indeed as advertised for Watson-Crick base-pairs where the minor groove side of the two bases in a pair aligns (facing the same direction). Specifically, in such cases, x-axis points from minor groove side to the major groove side, and y-axis points to the leading strand, and z-axis completes a right-handed coordinate system and would point in the 5' to 3' direction of the leading strand. As always, this is best illustrated with an example, and I am using bdl084.pdb, which is distributed with 3DNA. Try to reproduce the following:
find_pair bdl084.pdb bdl084.inp
frame_mol -f ref_frames.dat frame.pdb frame.alc
block_atom bdl084.pdb
comb_str bdl084.alc frame.alc bdl084_frame.alc
rasmol -alchemy -noconnect bdl084_frame.alc
Please note that there is a bug w.r.t. displaying Alchemy file in RasMol 2.7.x. I've been using v2.6.4, the version directly from Roger Sayle, the original author of RasMol.
Secondly, when a pair is no-longer of Watson-Crick type, the minor groove side of the two bases in a pair could be in quite different relative orientation. In such cases, where would the minor groove side of the pair be? And this is where you noticed "the axes appear to be pointing in these strange directions". Check carefully the base-pair parameters from the 3DNA main output file, and you would see the point. The "strange directions" are actually not that strange in that they are also the "middle" base-pair frame, as in the Watson-Crick pairs.
HTH,
Xiang-Jun