How about using the least squares method? If I understand correctly, your problem could be expressed as
P(t) = [A(t), B(t)] * [a; -b]
.
Let [a; -b] = x
, [A(t), B(t)] = Y
and P(t) = P
Now the least squares solution would be:
x = ((Y'*Y)^-1)*Y'*P;
In Matlab you could also use the 'backslash operator' for this case:
x = Y\P;
for this, you'll find the documentation here: mldivide
As a reference:
Wikipedia
Mathworks
I hope this helps.
EDIT:
Here's my test code:
A = [1;2;3]
B = [4;5;6]
P = [7;8;9]
Y = [A, -B]
disp('------- regular least squares formula -------')
x = ((Y'*Y)^-1)*Y'*P
a = x(1)
b = x(2)
disp('------- mldivide -------')
x = Y\P
a = x(1)
b = x(2)