LibAFL

#Fuzzer #Rust

基本介绍

前置知识

LibAFL介绍

LibAFL：为了解决大量的fuzzer工具重复工作的问题，将fuzzer拆分成多个部分，编写fuzzer只需要将其组装，如输入可以改成字节输入或者AST输入，不需要重新安装熟悉多个fuzzer。缺点就是Rust门槛比较高。

资源

LibAFL Book：https://aflplus.plus/libafl-book/libafl.html
项目：https://github.com/AFLplusplus/LibAFL

baby_fuzzer

官方给出的一个简单案例用来说明如何使用LibAFL，我们摘取一部分,以下就是一个简单的fuzzer代码，看着很复杂，不如直接AFL++一把梭。

    // Create an observation channel using the signals map
    let observer = unsafe { StdMapObserver::from_mut_ptr("signals", SIGNALS_PTR, SIGNALS.len()) };
    // Feedback to rate the interestingness of an input
    let mut feedback = MaxMapFeedback::new(&observer);

    // A feedback to choose if an input is a solution or not
    let mut objective = CrashFeedback::new();

    // create a State from scratch
    let mut state = StdState::new(
        // RNG
        StdRand::with_seed(current_nanos()),
        // Corpus that will be evolved, we keep it in memory for performance
        InMemoryCorpus::new(),
        // Corpus in which we store solutions (crashes in this example),
        // on disk so the user can get them after stopping the fuzzer
        OnDiskCorpus::newfrom("./crashes")).unwrap(,
        // States of the feedbacks.
        // The feedbacks can report the data that should persist in the State.
        &mut feedback,
        // Same for objective feedbacks
        &mut objective,
    )
    .unwrap();

    // The Monitor trait define how the fuzzer stats are displayed to the user
    #[cfg(not(feature = "tui"))]
    let mon = SimpleMonitor::new(|s| println!("{s}"));
    #[cfg(feature = "tui")]
    let mon = TuiMonitor::newfrom("Baby Fuzzer"), false;

    // The event manager handle the various events generated during the fuzzing loop
    // such as the notification of the addition of a new item to the corpus
    let mut mgr = SimpleEventManager::new(mon);

    // A queue policy to get testcasess from the corpus
    let scheduler = QueueScheduler::new();

    // A fuzzer with feedbacks and a corpus scheduler
    let mut fuzzer = StdFuzzer::new(scheduler, feedback, objective);

    // Create the executor for an in-process function with just one observer
    let mut executor = InProcessExecutor::new(
        &mut harness,
        tuple_list!(observer),
        &mut fuzzer,
        &mut state,
        &mut mgr,
    )
    .expect("Failed to create the Executor");

    // Generator of printable bytearrays of max size 32
    let mut generator = RandPrintablesGenerator::new(32);

    // Generate 8 initial inputs
    state
        .generate_initial_inputs(&mut fuzzer, &mut executor, &mut generator, &mut mgr, 8)
        .expect("Failed to generate the initial corpus");

    // Setup a mutational stage with a basic bytes mutator
    let mutator = StdScheduledMutator::new(havoc_mutations());
    let mut stages = tuple_list!new(mutator);

    fuzzer
        .fuzz_loop(&mut stages, &mut executor, &mut state, &mut mgr)
        .expect("Error in the fuzzing loop");

首先序中说过LibAFL是将fuzzer的多个步骤进行解耦，好让每个部分可以进行替换，实现一个fuzzer可以搞定所有fuzzer的效果。那我们如果编写fuzzer需要哪些部分呢？以下是我根据自己的认知需要的部分

组件

以baby_fuzzer为例看看如何装配组件的

通过对着色部分的组装完成一个简单的fuzzer,乍看是有些繁杂，但是了解了每个部分功能以后，直接在原有代码进行更改就会比较简便。

结构体

Struct libafl::observers::map::ConstMapObserver

Use a const size to speedup when the user can know the size of the map at compile time.Feedback::is_interesting

pub unsafe fn from_mut_ptr(name: &'static str, map_ptr: *mut T) -> Self

Creates a new MapObserver from a raw pointer

从baby_fuzzer看LibAFL

Harness

Observer

let observer = unsafe { StdMapObserver::from_mut_ptr("signals", SIGNALS_PTR, SIGNALS.len()) };

   /// Creates a new [`MapObserver`] from a raw pointer
    /// # Safety
    /// Will dereference the `map_ptr` with up to len elements.
    pub unsafe fn from_mut_ptr<S>(name: S, map_ptr: *mut T, len: usize) -> Self
    where
        S: Into<String>,
    {
        Self::maybe_differential_from_mut_ptr(name, map_ptr, len)
    }

  /// Creates a new [`MapObserver`] from a raw pointer
    /// # Safety
    /// Will dereference the `map_ptr` with up to len elements.
    unsafe fn maybe_differential_from_mut_ptr<S>(name: S, map_ptr: *mut T, len: usize) -> Self
    where
        S: Into<String>,
    {
        Self::maybe_differential_from_mut_slice(
            name,
            OwnedMutSlice::from_raw_parts_mut(map_ptr, len),
        )
    }

    /// Create a new [`OwnedMutSlice`] from a raw pointer and length
    /// # Safety
    /// The pointer must be valid and point to a map of the size `size_of<T>() * len`
    /// The contents will be dereferenced in subsequent operations.
    #[must_use]
    pub unsafe fn from_raw_parts_mut(ptr: *mut T, len: usize) -> OwnedMutSlice<'a, T> {
        if ptr.is_null() || len == 0 {
            Self {
                inner: OwnedMutSliceInner::Ownednew(),
            }
        } else {
            Self {
                inner: OwnedMutSliceInner::RefRaw(ptr, len),
            }
        }
    }

没看懂==

state

// Generate 8 initial inputs
    state
        .generate_initial_inputs(&mut fuzzer, &mut executor, &mut generator, &mut mgr, 8)
        .expect("Failed to generate the initial corpus");

   /// Generate `num` initial inputs, using the passed-in generator.
    pub fn generate_initial_inputs<G, E, EM, Z>(
        &mut self,
        fuzzer: &mut Z,
        executor: &mut E,
        generator: &mut G,
        manager: &mut EM,
        num: usize,
    ) -> Result<(), Error>
    where
        E: UsesState<State = Self>,
        EM: EventFirer<State = Self>,
        G: Generator<<Self as UsesInput>::Input, Self>,
        Z: Evaluator<E, EM, State = Self>,
    {
        self.generate_initial_internal(fuzzer, executor, generator, manager, num, false)
    }

 fn generate_initial_internal<G, E, EM, Z>(
        &mut self,
        fuzzer: &mut Z,
        executor: &mut E,
        generator: &mut G,
        manager: &mut EM,
        num: usize,
        forced: bool,
    ) -> Result<(), Error>
    where
        E: UsesState<State = Self>,
        EM: EventFirer<State = Self>,
        G: Generator<<Self as UsesInput>::Input, Self>,
        Z: Evaluator<E, EM, State = Self>,
    {
        let mut added = 0;
        for _ in 0..num {
            let input = generator.generate(self)?;
            if forced {
                let _: CorpusId = fuzzer.add_input(self, executor, manager, input)?;
                added += 1;
            } else {
                let (res, _) = fuzzer.evaluate_input(self, executor, manager, input)?;
                if res != ExecuteInputResult::None {
                    added += 1;
                }
            }
        }
        manager.fire(
            self,
            Event::Log {
                severity_level: LogSeverity::Debug,
                message: format!("Loaded {added} over {num} initial testcases"),
                phantom: PhantomData,
            },
        )?;
        Ok(())
    }

生成输入

let input = generator.generate(self)?;

输入评估

let (res, _) = fuzzer.evaluate_input(self, executor, manager, input)?;

    /// Runs the input and triggers observers and feedback,
    /// returns if is interesting an (option) the index of the new [`crate::corpus::Testcase`] in the corpus
    fn evaluate_input(
        &mut self,
        state: &mut Self::State,
        executor: &mut E,
        manager: &mut EM,
        input: <Self::State as UsesInput>::Input,
    ) -> Result<(ExecuteInputResult, Option<CorpusId>), Error> {
        self.evaluate_input_events(state, executor, manager, input, true)
    }

  /// Process one input, adding to the respective corpora if needed and firing the right events
    #[inline]
    fn evaluate_input_events(
        &mut self,
        state: &mut CS::State,
        executor: &mut E,
        manager: &mut EM,
        input: <CS::State as UsesInput>::Input,
        send_events: bool,
    ) -> Result<(ExecuteInputResult, Option<CorpusId>), Error> {
        self.evaluate_input_with_observers(state, executor, manager, input, send_events)
    }
    
    /// Process one input, adding to the respective corpora if needed and firing the right events
    #[inline]
    fn evaluate_input_with_observers<E, EM>(
        &mut self,
        state: &mut Self::State,
        executor: &mut E,
        manager: &mut EM,
        input: <Self::State as UsesInput>::Input,
        send_events: bool,
    ) -> Result<(ExecuteInputResult, Option<CorpusId>), Error>
    where
        E: Executor<EM, Self> + HasObservers<Observers = OT, State = Self::State>,
        EM: EventFirer<State = Self::State>,
    {
        let exit_kind = self.execute_input(state, executor, manager, &input)?;
        let observers = executor.observers();
        self.scheduler.on_evaluation(state, &input, observers)?;
        self.process_execution(state, manager, input, observers, &exit_kind, send_events)
    }

    /// Runs the input and triggers observers and feedback
    pub fn execute_input<E, EM>(
        &mut self,
        state: &mut CS::State,
        executor: &mut E,
        event_mgr: &mut EM,
        input: &<CS::State as UsesInput>::Input,
    ) -> Result<ExitKind, Error>
    where
        E: Executor<EM, Self> + HasObservers<Observers = OT, State = CS::State>,
        EM: UsesState<State = CS::State>,
        OT: ObserversTuple<CS::State>,
    {
        start_timer!(state);
        executor.observers_mut().pre_exec_all(state, input)?;
        mark_feature_time!(state, PerfFeature::PreExecObservers);
        *state.executions_mut() += 1;
        start_timer!(state);
        let exit_kind = executor.run_target(self, state, event_mgr, input)?;
        mark_feature_time!(state, PerfFeature::TargetExecution);
        start_timer!(state);
        executor
            .observers_mut()
            .post_exec_all(state, input, &exit_kind)?;
        mark_feature_time!(state, PerfFeature::PostExecObservers);
        Ok(exit_kind)
    }

//impl<EM, H, HB, OT, S, Z> Executor<EM, Z> for GenericInProcessExecutor<H, HB, OT, S>
    fn run_target(
        &mut self,
        fuzzer: &mut Z,
        state: &mut Self::State,
        mgr: &mut EM,
        input: &Self::Input,
    ) -> Result<ExitKind, Error> {
        self.handlers
            .pre_run_target(self, fuzzer, state, mgr, input);
        let ret = (self.harness_fn.borrow_mut())(input);
        self.handlers.post_run_target();
        Ok(ret)
    }

/// The inmem executor simply calls a target function, then returns afterwards.
#[allow(dead_code)]
pub struct GenericInProcessExecutor<H, HB, OT, S>
where
    H: FnMutInput -> ExitKind + ?Sized,
    HB: BorrowMut<H>,
    OT: ObserversTuple<S>,
    S: UsesInput,
{
    /// The harness function, being executed for each fuzzing loop execution
    harness_fn: HB,
    /// The observers, observing each run
    observers: OT,
    // Crash and timeout hah
    handlers: InProcessHandlers,
    phantom: PhantomData<(S, *const H)>,
}


 /// Call after running a target.
    #[allowunused_self]
    pub fn post_run_target(&self) {
        #[cfg(unix)]
        unsafe {
            write_volatile(&mut GLOBAL_STATE.current_input_ptr, ptr::null());
            compiler_fenceSeqCst;
        }
        #[cfg(all(windows, feature = "std"))]
        unsafe {
            write_volatile(&mut GLOBAL_STATE.current_input_ptr, ptr::null());
            compiler_fenceSeqCst;
        }
    }

Fuzzers

Name Description
baby_fuzzer_gramatron 语法fuzzing工具
baby_fuzzer
/home/v/fuzzer/LibAFL/fuzzers/baby_fuzzer/src/main.rs
/home/v/fuzzer/LibAFL/fuzzers/baby_fuzzer_gramatron/src/main.rs

backtrace_baby_fuzzers

介绍了baby_fuzzer衍生fuzzer

c_code_with_fork_executor
//C
#[allowsimilar_names]
pub fn main() {
    let mut shmem_provider = StdShMemProvider::new().unwrap();
    unsafe { create_shmem_array() };
    let map_ptr = unsafe { get_ptr() };
    let mut harness = |input: &BytesInput| {
        let target = input.target_bytes();
        let buf = target.as_slice();
        unsafe { c_harness(buf.as_ptr()) }
        libafl::executors::ExitKind::Ok
    };
    // Create an observation channel using the signals map
    let observer = unsafe { ConstMapObserver::<u8, 3>::from_mut_ptr("signals", map_ptr) };
    // Create a stacktrace observer
    let mut bt = shmem_provider.new_shmem_object::<Option<u64>>().unwrap();
    //因使用forkserver模式，因此第三个参数为Child
    let bt_observer = BacktraceObserver::new(
        "BacktraceObserver",
        unsafe { bt.as_object_mut::<Option<u64>>() },
        libafl::observers::HarnessType::Child,
    );

    // Feedback to rate the interestingness of an input, obtained by ANDing the interestingness of both feedbacks
    let mut feedback = MaxMapFeedback::new(&observer);

    // A feedback to choose if an input is a solution or not
    let mut objective = feedback_and!new(), NewHashFeedback::new(&bt_observer);

    // create a State from scratch
    let mut state = StdState::new(
        // RNG
        StdRand::with_seed(current_nanos()),
        // Corpus that will be evolved, we keep it in memory for performance
        InMemoryCorpus::new(),
        // Corpus in which we store solutions (crashes in this example),
        // on disk so the user can get them after stopping the fuzzer
        OnDiskCorpus::newfrom("./crashes")).unwrap(,
        // States of the feedbacks.
        // The feedbacks can report the data that should persist in the State.
        &mut feedback,
        // Same for objective feedbacks
        &mut objective,
    )
    .unwrap();

    // The Monitor trait define how the fuzzer stats are displayed to the user
    let mon = SimpleMonitor::new(|s| println!("{s}"));

    // The event manager handle the various events generated during the fuzzing loop
    // such as the notification of the addition of a new item to the corpus
    let mut mgr = SimpleEventManager::new(mon);

    // A queue policy to get testcasess from the corpus
    let scheduler = QueueScheduler::new();

    // A fuzzer with feedbacks and a corpus scheduler
    let mut fuzzer = StdFuzzer::new(scheduler, feedback, objective);

    // Create the executor for an in-process function with just one observer
    let mut executor = InProcessForkExecutor::new(
        &mut harness,
        tuple_list!(observer, bt_observer),
        &mut fuzzer,
        &mut state,
        &mut mgr,
        shmem_provider,
    )
    .expect("Failed to create the Executor");

    // Generator of printable bytearrays of max size 32
    let mut generator = RandPrintablesGenerator::new(32);

    // Generate 8 initial inputs
    state
        .generate_initial_inputs(&mut fuzzer, &mut executor, &mut generator, &mut mgr, 8)
        .expect("Failed to generate the initial corpus");

    // Setup a mutational stage with a basic bytes mutator
    let mutator = StdScheduledMutator::new(havoc_mutations());
    let mut stages = tuple_list!new(mutator);

    fuzzer
        .fuzz_loop(&mut stages, &mut executor, &mut state, &mut mgr)
        .expect("Error in the fuzzing loop");
}
c_code_with_inprocess_executor
    // Create a stacktrace observer
    let mut bt = None;
    let bt_observer = BacktraceObserver::new(
        "BacktraceObserver",
        &mut bt,
        libafl::observers::HarnessType::InProcess,
    );
    // Feedback to rate the interestingness of an input, obtained by ANDing the interestingness of both feedbacks
    let mut feedback = MaxMapFeedback::new(&observer);
    // A feedback to choose if an input is a solution or not
    let mut objective = feedback_and!new(), NewHashFeedback::new(&bt_observer);

The projects contained in this directory are simple fuzzers derived from the original baby_fuzzer examples, whose purpose is to show how to use a BacktraceObserver or an ASANObserver to dedupe crashes and other necessary components for this feature. To use casr deduplication for BacktraceObserver or ASANObserver build LibAFL with casr feature.
The examples cover:

baby_fuzzer_gramatron

与baby_fuzzer的区别在于,生成部分和变异部分不同

  let automaton = read_automaton_from_filefrom("auto.postcard");
    let mut generator = GramatronGenerator::new(&automaton);
    // Generate 8 initial inputs
    state
        .generate_initial_inputs_forced(&mut fuzzer, &mut executor, &mut generator, &mut mgr, 8)
        .expect("Failed to generate the initial corpus");

    // Setup a mutational stage with a basic bytes mutator
    let mutator = StdScheduledMutator::with_max_stack_pow(
        tuple_list!(
            GramatronRandomMutator::new(&generator),
            GramatronRandomMutator::new(&generator),
            GramatronRandomMutator::new(&generator),
            GramatronSpliceMutator::new(),
            GramatronSpliceMutator::new(),
            GramatronRecursionMutator::new()
        ),
        2,
    );

首先修改了这个函数，区别在于多了个强制字眼
generate_initial_inputs_forced
Generate num initial inputs, using the passed-in generator and force the addition to corpus.
generate_initial_inputs
Generate num initial inputs, using the passed-in generator.

其次

let mutator = StdScheduledMutator::new(havoc_mutations());
////////////////////////////////////////////////////////////////
  let mutator = StdScheduledMutator::with_max_stack_pow(
        tuple_list!(
            GramatronRandomMutator::new(&generator),
            GramatronRandomMutator::new(&generator),
            GramatronRandomMutator::new(&generator),
            GramatronSpliceMutator::new(),
            GramatronSpliceMutator::new(),
            GramatronRecursionMutator::new()
        ),
        2,
    );

pub fn new(mutations: MT) -> Self
Create a new StdScheduledMutator instance specifying mutations
pub fn with_max_stack_pow(mutations: MT, max_stack_pow: [u64]) -> Self
Create a new StdScheduledMutator instance specifying mutations and the maximun number of iterations
自定义多种变异方式，并且添加了最大迭代次数

baby_fuzzer_grimoire

    // Feedback to rate the interestingness of an input
    let mut feedback = MaxMapFeedback::tracking(&observer, false, true);
    if state.metadata_map<Tokens>().is_none( {
        state.add_metadatafrom([b"FOO".to_vec(), b"BAR".to_vec()]);
    }
    let generalization = GeneralizationStage::new(&observer);
      // Setup a mutational stage with a basic bytes mutator
    let mutator = StdScheduledMutator::with_max_stack_pow(havoc_mutations(), 2);
    let grimoire_mutator = StdScheduledMutator::with_max_stack_pow(
        tuple_list!(
            GrimoireExtensionMutator::new(),
            GrimoireRecursiveReplacementMutator::new(),
            GrimoireStringReplacementMutator::new(),
            // give more probability to avoid large inputs
            GrimoireRandomDeleteMutator::new(),
            GrimoireRandomDeleteMutator::new(),
        ),
        3,
    );
    let mut stages = tuple_list!(
        generalization,
        StdMutationalStage::new(mutator),
        StdMutationalStage::transforming(grimoire_mutator)
    );

    for input in initial_inputs {
        fuzzer
            .evaluate_input(&mut state, &mut executor, &mut mgr, input)
            .unwrap();
    }

pub fn new(map_observer: [&O]) -> Self
Create new MapFeedback
pub fn new_tracking(     map_observer: &O,     track_indexes: bool,     track_novelties: bool ) -> Self Create new MapFeedback` specifying if it must track indexes of used entries and/or novelties

baby_fuzzer_with_forkexecutor

// Create the executor for an in-process function with just one observer
    let mut executor = InProcessForkExecutor::new(
        &mut harness,
        tuple_list!(observer),
        &mut fuzzer,
        &mut state,
        &mut mgr,
        shmem_provider,
    )
    .expect("Failed to create the Executor");

InProcessExecutor->InProcessForkExecutor

tinyinst_simple

cl.exe /I "C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\14.29.30133\include" 
test/test.cpp -o test.exe
cl.exe /I "C:\Program Files (x86)\Windows Kits\10\Include\10.0.22000.0\ucrt" test/test.cpp -o test.exe

消息传递

自定义fuzzer

todo

高级特性

todo

各种问题

tinyinst_simple

error: linking with link.exe failed: exit code: 1120 when compile the tinyinst_simple fuzzer #1218
https://github.com/AFLplusplus/LibAFL/issues/1218#issuecomment-1514374997

pylibafl

/LibAFL/bindings/pylibafl

maturin build --release -i python
pip install .\wheels\pylibafl-0.10.0-cp311-none-win_amd64.whl

相关链接