Martensitic filler metals with low transformation temperatures can efficiently mitigate harmful tensile welding residual stress. It is vital to clarify the difference in the influence of martensitic transformation for different shapes of welded joints. This article outlines the influence of martensitic transformation in a butt-welded plate and a butt-welded pipe which were designed to have the same dimensions of the cross-section perpendicular to weld line. A clear difference in the influence of martensitic transformation was found in these two joints. Longitudinal tensile stress in weld zone is efficiently reduced in both joints, whereas longitudinal tensile stress is formed in base metal near the weld zone in the pipe. A notably greater influence on transverse stress is found in the pipe than in the plate