@SteveThompson
AI trained on widely available data is going to do a good job of emulating the moral decisions in that data.
If the participants rating it are ordinary folks rather than, say, ethicists, a high rating likely means it's good at emulating the responses they like, not necessarily that its responses are more moral, ethical, or just.
In other words, it could be that they like the training data, which might or might not be biased, and it's good at reproducing the training data.