+
Skip to content

Can certain map functions reduce allocations by using f(x) to calculate df(x) in backward? #69

@coreylowman

Description

@coreylowman

This is now possible with Rc usage in #62.

Something like:

pub fn map_dff<T: Tensor<Dtype = f32>, F, Df>(mut t: T, mut f: F, mut df: Df) -> T
where
    F: 'static + FnMut(&f32) -> f32,
    Df: 'static + FnMut(&f32) -> f32,
{
    T::Device::foreach_m(t.mut_data(), &mut |x| *x = f(x));
    let (t, mut tape) = t.split_tape();
    let result = t.clone(); // should add a new reference, not start a new one
    let phantom_result = result.phantom();
    tape.add_backward_op(move |grads| {
        let (t_grad, result_grad) = grads.mut_and_ref(&t, &phantom_result);
        T::Device::foreach_mrr(t_grad, t.data(), result_grad, &mut |g, t, r| {
            *g += df(t) * r;
        });
    });
    result.put_tape(tape)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      点击 这是indexloc提供的php浏览器服务,不要输入任何密码和下载