This is how I successfully converted a mkv file to a PS3 compatible mp4 file (running Ubuntu 8.04).
First you need to compile ffmpeg with neccessary flags, this guide will show you how.
Then use synaptics to install mkvtoolnix and mpeg4ip-server.
Finally a little script to convert the file:
#!/bin/bash
# converts mkv to mp4
# Check that we can read the input file
[ ! -r "$1" ] && echo "Error! Cannot read $1 !" && exit 1
# Get name and dir from the path
bname=`basename $1`
dname=`dirname $1`
# Dump track information
mkvinfo "$1" | grep -i track > output.txt
# Parse formats
f=`sed -n '/Track type/p' output.txt | sed 's/^.*://' | sed 's/^ *//'`
# Parse track numbers
n=`sed -n '/Track number/p' output.txt | sed 's/^.*://' | sed 's/^ *//'`
# Parse speeds (fps)
s=`sed -n '/Default duration/p' output.txt | sed 's/^.*(//' | sed 's/fps.*)//'`
# Remove temporary file
rm output.txt
# Put track numbers in an array
x=0
for i in $n
do
number[$x]=$i
x=$x+1
done
# Put speeds in an array
x=0
for i in $s
do
fps[$x]=$i
x=$x+1
done
# Match track speeds with formats
x=0
for i in $f
do
if [ "video" = $i ]; then
vidtrack=${number[x]}
vidfps=${fps[x]}
fi
if [ "audio" = $i ]; then
audtrack=${number[x]}
fi
x=$x+1
done
echo "Extracting tracks from mkv file"
mkvextract tracks $1 ${vidtrack}:video.h264
ffmpeg -i $1 -sameq -vn -acodec libfaac -ar 48000 -ab 320k audio.aac
echo "Putting it all in a new and shiny mp4 container"
mp4creator -create=video.h264 -rate=$vidfps ${dname}/${bname}.mp4
mp4creator -c audio.aac -interleave -optimize ${dname}/${bname}.mp4
rm audio.aac video.h264