Laravel 9 Upload Large CSV File Using Queue


In this article, we will see laravel 9 upload a large CSV file using a queue. Here we will learn how to upload a large CSV file in the database using a queue in laravel 8 and laravel 9. Sometimes we need to insert large numbers of records into the database using the queue job in laravel 9.

Also, we use queue jobs in laravel 9 for the background process. When uploading more than 50K+ records to the database. So, it takes more loading time to insert into the database. For that reason, we will use the laravel queue job. If we use a queue job all processes can background in our application and it can improve or boost the performance.

So, let's see import a large CSV file in laravel 9 using the queue, how to import a large CSV using the queue in laravel 8 and laravel 9, upload a large CSV file in laravel 9, and upload a large CSV file in the database using laravel 9.

Step 1: Install Laravel 9

In this step, we will install the laravel 9 application using the following command.

composer create-project --prefer-dist laravel/laravel laravel-9-csv-upload

 

 

 Step 2: Create Model and Migration

Now, we will create a model and migration using the following command.

php artisan make:model Item -m

app/Models/Item.php

<?php
 
namespace App\Models;
 
use Illuminate\Database\Eloquent\Factories\HasFactory;
use Illuminate\Database\Eloquent\Model;
 
class Item extends Model
{
    use HasFactory;
    protected $guarded = [];
}

Migration:

public function up()
{
    Schema::create('items', function (Blueprint $table) {
        $table->bigIncrements('id');
        $table->string('name');
        $table->string('sku');
        $table->text('description');
        $table->string('price');
        $table->timestamps();
    });
}

Now, run the migration using the below command.

php artisan migrate

 

Step 3: Create Route

Now, we will create routes for the file upload form, and then we have to upload it to the server. So, add the below code to the web.php file.

routes/web.php

use Illuminate\Support\Facades\Route;
use App\Http\Controllers\ItemController;

/*
|--------------------------------------------------------------------------
| Web Routes
|--------------------------------------------------------------------------
|
| Here is where you can register web routes for your application. These
| routes are loaded by the RouteServiceProvider within a group which
| contains the "web" middleware group. Now create something great!
|
*/

Route::get('/', [ItemController::class,'index']);
Route::post('/', [ItemController::class,'upload_csv_file'])->name('upload');

 

Step 4: Create Controller

In this step, we will create an ItemController.php file. So, add the following code to that file. 

app/Http/Controllers/ItemController.php

namespace App\Http\Controllers;

use Illuminate\Http\Request;
use App\Models\Item;
use App\Jobs\ItemCSVUploadJob;

class ItemController extends Controller
{
    public function index()
    {
        return view('index');
    }

    public function upload_csv_file(Request $request)
    {
        if( $request->has('csv') ) {

            $csv    = file($request->csv);
            $chunks = array_chunk($csv,1000);
            $header = [];

            foreach ($chunks as $key => $chunk) {
            $data = array_map('str_getcsv', $chunk);
                if($key == 0){
                    $header = $data[0];
                    unset($data[0]);
                }

                ItemCSVUploadJob::dispatch($data, $header);                
            }

        }
        return "please upload CSV file";
    }
}

 

 

Step 5: Create a View

In this step, we will create a blade view to show our file upload form. So, create it to complete the laravel CSV file upload example with the queue job.

resources/views/index.blade.php


<!DOCTYPE html>
<html lang="{{ str_replace('_', '-', app()->getLocale()) }}">
    <head>
        <meta charset="utf-8">
        <meta name="viewport" content="width=device-width, initial-scale=1">
        <title>Laravel 9 Upload Large CSV File Using Queue - Techsolutionstuff</title>
    </head>
    <body class="antialiased">
        <div class="relative flex items-top justify-center min-h-screen bg-gray-100 dark:bg-gray-900 sm:items-center py-4 sm:pt-0">

            <form action="{{ route('upload') }}" method="POST" enctype="multipart/form-data">
                @csrf
                <input type="file" name="csv">
                <input type="submit" value="submit">
          </form>

        </div>
    </body>
</html>

 

Step 6: Create a Job

In this step, we will create a job to process our large CSV file. So, run the following command.

php artisan make:job ItemCSVUploadJob

Now, update the ItemCSVUploadJob code like below.

app/Jobs/ItemCSVUploadJob.php

<?php

namespace App\Jobs;

use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use App\Models\Item;

class ItemCSVUploadJob implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
    
    public $header;
    public $data;

    /**
     * Create a new job instance.
     *
     * @return void
     */
    public function __construct($data, $header)
    {
        $this->data = $data;
        $this->header = $header;
    }

    /**
     * Execute the job.
     *
     * @return void
     */
    public function handle()
    {
        foreach ($this->data as $item) {
            $item_csv_data = array_combine($this->header,$item);
            Item::create($item_csv_data);
        }
    }
}

 

 

 Step 7: Create Queue Table

Now, we will create a 'jobs' table in the database. So, copy the below command and run it in your terminal.

php artisan queue:table

php artisan migrate‚Äč

 After uploading CSV File run the below command in your terminal to upload CSV files into the database.

php artisan queue:listen

 


You might also like:

RECOMMENDED POSTS

FEATURE POSTS